Question regarding Interpreting Coefficients

[Ignacio Lunghi]

Hi,

The problem I have is the following. I have ran two regressions in Stata:

(1) “reg income i.treated##i.male i.experiment”

(2) “reg income i.treated i.experiment if male==1”

Where treated and male are two dummies and experiment is a categorical variable. The income variable is continuous.

I thought that the sum of the coefficients "_b[1.p treated] + _b[1.ptreated#1.man]" of the first regression and the coefficient "_b[1.ptreated]" of the second were the same, but they are not.

What explanation is there for this?

Additional information: the sum of the coefficients is different only when I incorporate dummies per experiment (I had to add them since we are grouping different experiment which were done using Qualtrics). They are equal when i have ran the following regressions.

(3) “reg income i.treated##i.male”

(4) “reg income i.treated if male==1”

Is there a way I can perform a regression like the first one but where the sum of "_b[1.ptreated] +_b[1.ptreated#1.male]" and the coefficient "_b[1.ptreated]" are equal between regressions?

Here are the results of the regressions:

	(1)	(2)	(3)	(4)
Variables	income	income	income	income
1.treated	0.0402	0.0370	0.0774*	0.0699
	(0.0438)	(0.0511)	(0.0434)	(0.0494)
1.male	0.121**		0.125**
	(0.0490)		(0.0493)
1.treated#1.male	-0.00707		-0.00750
	(0.0639)		(0.0642)
Experiment 2	-0.0920**	-0.0798
	(0.0361)	(0.0561)
Experiment 3	-0.285***	-0.263***
	(0.0441)	(0.0671)
Constant	-0.0515	0.0576	-0.172***	-0.0469
	(0.0422)	(0.0543)	(0.0333)	(0.0381)
Observations	3,771	1,731	3,771	1,731
R-squared	0.016	0.010	0.005	0.001
Standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1

[Clyde Schechter]

I thought that the sum of the coefficients "_b[1.p treated] + _b[1.ptreated#1.man]" of the first regression and the coefficient "_b[1.ptreated]" of the second were the same, but they are not.

What explanation is there for this?

Additional information: the sum of the coefficients is different only when I incorporate dummies per experiment (I had to add them since we are grouping different experiment which were done using Qualtrics). They are equal when i have ran the following regressions.

You have actually answered your own question.

Stratified regression and interacting a single regression with the stratified variable are not exactly equivalent. They do produce the same result when only a single explanatory variable is present in the model. But when the regression involves multiple predictors and you interact only with one, you will get different results. To get the same results from interaction as with stratification you must interact with all of the explanatory variables. In your case:

reg income i.treated i.experiment if male == 1
reg income i.male##(i.treated i.experiment)

will be have the way you expect.

The reason -reg income i.male##i.treated i.experiment- doesn't work the way you expected it to is that when you separately regress for males and females, the experiment variable is allowed to have different values for males and females (and, except in extraordinary coincidental circumstances, it almost always will). By contrast when you interact male with treated, but not with experiment, you constrain the coefficient of treated to be the same for males and females, while that for treated can (and almost always will) differ. But unless treated and experiment are exactly independent variables, anything that changes the coefficient of one of them will also change the coefficient of the other.

[Ignacio Lunghi]

Ok, thank you very much for the answer

I also understand, based on what you told me, that the most "correct" way in this experiment would then be to add those interactions between the "experiment" variable and my sex and treatment variables, right?

[Clyde Schechter]

Well, I didn't say that, and I don't know what the most "correct" approach is here. What I said is that adding those interactions will cause the interaction model coefficients to equal those you get from doing separate regressions for each sex.

But it is a substantive, not a statistical, question whether the correct approach is to have the effect of experiment differ between men and women or constrain it to be the same. That would depend on what the experiments are, and whether predicts that those effects should be the same or different. It's really no different from any other situation where you are trying to decide whether or not to include an interaction term in a model: it depends on what is understood about the data generating process and whether it distinguishes those groups or not. The question you are posing in #3 requires advice from somebody with expertise in the subject matter of your research--it is not a statistical question.

I can't tell from your posts what discipline you are working in: on the one hand you are at least considering sex-specific differences, which is mostly characteristic of the biomedical disciplines, but you are also modeling income as the outcome variable, suggesting an economics related discipline. I'm an epidemiologist and I wouldn't pretend to advise you on an economics-related question. If your work is sufficiently closely related to areas I have worked in, I might be able to advise you if you provided more detail about what you are studying and what these experiments involve.

[Ignacio Lunghi]

Well, I didn't say that, and I don't know what the most "correct" approach is here. What I said is that adding those interactions will cause the interaction model coefficients to equal those you get from doing separate regressions for each sex.

But it is a substantive, not a statistical, question whether the correct approach is to have the effect of experiment differ between men and women or constrain it to be the same. That would depend on what the experiments are, and whether predicts that those effects should be the same or different. It's really no different from any other situation where you are trying to decide whether or not to include an interaction term in a model: it depends on what is understood about the data generating process and whether it distinguishes those groups or not. The question you are posing in #3 requires advice from somebody with expertise in the subject matter of your research--it is not a statistical question.

I can't tell from your posts what discipline you are working in: on the one hand you are at least considering sex-specific differences, which is mostly characteristic of the biomedical disciplines, but you are also modeling income as the outcome variable, suggesting an economics related discipline. I'm an epidemiologist and I wouldn't pretend to advise you on an economics-related question. If your work is sufficiently closely related to areas I have worked in, I might be able to advise you if you provided more detail about what you are studying and what these experiments involve.

Yes, I thought that the answer was related more to the data generating process. Thank you, I'm going to look at it with more detail. I greatly appreciate your help!