'Wrong' sign with fixed effects

[Janet Lewis]

Hi there,

I was hoping to get help regarding a change is signs of a coefficient when using fixed effects regressions vs random effects.

I am measuring changes in women's life satisfaction with partner's presence of a health condition as a control. The coefficient on this variable is positive when running fixed effect regressions. This seems odd, as one would assume that women living with a partner who has a health condition will reduce their overall life satisfaction. To try and determine whether this has something to do with correlation between other controls I removed them step by step. Still even if I regress something like this with no other controls including year dummies:

xtreg satisfaction partner_health_condition, fe robust

I get a positive coefficient on partner_health_condition. Though if I run something like this:

reg satisfaction partner_health_condition, robust

The coefficient on partner_health_condition is negative as expected. I am struggling to get my head around this. I have run a Hausman test and FE is the appropriate estimation method.
I know this is not to do with an error in my coding of the variables as I do not have this issue when running it on the male sample.

Your help would be greatly appreciated!

[Carlo Lazzaro]

Janet:
you're comparing two different (and not interchangeable) regression models: no wonder that you got different signs.
That said, your chances of getting more helpful replies are conditional on posting what you typed (as you did) and what Stata gave you back (as per FAQ). Thanks.

[Janet Lewis]

Thank you for getting back to me Carlo. Can I ask what you mean by the two regressions are different, yes I understand one has fixed effects estimates while the other does not. In other word, the fe regressions will provide within-individual comparisons while the re regressions provide between individual comparisons. An I missing something?
I am posting output for the fixed effects model followed by the re models.

	

		

			

			
				Fixed-effects (within) regression               Number of obs     =     12,608
Group variable: xwaveid                         Number of groups  =      3,140

R-sq:                                           Obs per group:
     within  = 0.0003                                         min =          1
     between = 0.0036                                         avg =        4.0
     overall = 0.0026                                         max =         13

                                                F(1,3139)         =       2.55
corr(u_i, Xb)  = -0.0836                        Prob > F          =     0.1107

                            (Std. Err. adjusted for 3,140 clusters in xwaveid)
------------------------------------------------------------------------------
             |               Robust
satisfaction |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
  ptn_health |   .0544205   .0341072     1.60   0.111    -.0124541     .121295
       _cons |   8.038507   .0048964  1641.71   0.000     8.028907    8.048108
-------------+----------------------------------------------------------------
     sigma_u |  1.0576008
     sigma_e |  .83322739
         rho |  .61701638   (fraction of variance due to u_i)
------------------------------------------------------------------------------
Linear regression                    Number of obs     =     12,608
                                                F(1, 3139)        =       9.66
                                                Prob > F          =     0.0019
                                                R-squared         =     0.0026
                                                Root MSE          =     1.1998

                            (Std. Err. adjusted for 3,140 clusters in xwaveid)
------------------------------------------------------------------------------
             |               Robust
satisfaction |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
  ptn_health |   -.174717   .0562272    -3.11   0.002    -.2849628   -.0644712
       _cons |   8.071402   .0199137   405.32   0.000     8.032357    8.110447
------------------------------------------------------------------------------
			
		
	

Sorry for the multiple edits! I was trying to get the output formatting right!

[Carlo Lazzaro]

Janet:
1) your -xtreg,fe- is in all likelihood misspecified, as your Rsq-within is dramatic low (by the way: you're right that -fe- estimator investigate time-varying variabiliy within the same panel, getting rid of time-invariant predictors, whereas -re- focuses on between-panel variation. In addition, -regress- uses a mix of within and between variations).
2) your model would have given back the same resuts as far as the common coefficients (ie, constant, standard errors and related stuff would have remained different) had -regress- included -xtreg,fe- -panelid- among the predictors (please see the following toy-example, where -cluster- is used unappropriately, due to the limited number of panels):

. use "https://www.stata-press.com/data/r16/nlswork.dta"
(National Longitudinal Survey.  Young Women 14-26 years of age in 1968)

. xtreg ln_wage c.age##c.age if idcode<=3, fe vce(cluster idcode)

Fixed-effects (within) regression               Number of obs     =         39
Group variable: idcode                          Number of groups  =          3

R-sq:                                           Obs per group:
     within  = 0.6382                                         min =         12
     between = 0.8744                                         avg =       13.0
     overall = 0.2765                                         max =         15

                                                F(2,2)            =       3.83
corr(u_i, Xb)  = -0.2473                        Prob > F          =     0.2070

                                 (Std. Err. adjusted for 3 clusters in idcode)
------------------------------------------------------------------------------
             |               Robust
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .2512762   .1007559     2.49   0.130    -.1822416     .684794
             |
 c.age#c.age |  -.0037603   .0015163    -2.48   0.131    -.0102844    .0027638
             |
       _cons |  -2.189815   1.575348    -1.39   0.299    -8.967992    4.588361
-------------+----------------------------------------------------------------
     sigma_u |  .31366066
     sigma_e |  .19867104
         rho |  .71367959   (fraction of variance due to u_i)
------------------------------------------------------------------------------

. reg ln_wage i.idcode c.age##c.age if idcode<=3, vce(cluster idcode)

Linear regression                               Number of obs     =         39
                                                F(1, 2)           =          .
                                                Prob > F          =          .
                                                R-squared         =     0.7407
                                                Root MSE          =     .19867

                                 (Std. Err. adjusted for 3 clusters in idcode)
------------------------------------------------------------------------------
             |               Robust
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      idcode |
          2  |  -.4231615   .0288023   -14.69   0.005    -.5470877   -.2992353
          3  |  -.6126416   .0625166    -9.80   0.010    -.8816288   -.3436544
             |
         age |   .2512762    .103677     2.42   0.136    -.1948099    .6973623
             |
 c.age#c.age |  -.0037603   .0015603    -2.41   0.138    -.0104736     .002953
             |
       _cons |   -1.82398   1.588179    -1.15   0.370    -8.657361      5.0094
------------------------------------------------------------------------------

.

3) -robust- and -cluster- options do the very same jobs under -xtreg- but that does not hold for -regress- (see -regress- help file and/or related entry in Stata .pdf manual). Briefly, the -robust- option under -regress- takes only heteroskedasticity into account.

[Janet Lewis]

Thank you for taking time to explain this. Though, I probably used a bad example, in my complete model my predictors include number of children, age, age sq, Married, education, own presence of a health condition and partner's presence of a health condition, as well as panel dummies (I use i.year ). I assume by panelid you mean something like i.year? I ran the regressions without any other predicters simply to check for an issue of collinearity amongst predictors.
I am using restricted data, so I was a little weary about what I can and cannot post.

 xtreg satisfaction number_children age age_sq Married Education own_health_cond partner_health_cond i.wave, fe cluster(personid)

I get a reasonable R sq, though the coefficient on the partners presence of a health condition is positive. This isn't the case if I run a re model. I wonder if like you mentioned, I would be measuring something different in both estimation strategies. Perhaps, its something to do with dynamic confusion as in the text below. Or maybe still some misspecification in my code. Though I have run a similar model for other subgroups with no problem at all, and I have also used different domains of satisfaction as the independent variable. Still trying to understand this...
Thank you for yours clarification on robust and cluster options too!

http://www.stat.columbia.edu/~gelman...wrong_sign.pdf

[daniel klein]

Just a couple of remarks.

First, none of the code we have seen so far estimates a random-effects model; they estimate pooled linear regression models. The two are not the same. In Stata terms

regress ...

is not the same as

xtreg ... , re

Second, neither the pooled nor the random-effects model "focusses" on between-panel variation. Both models use the between-panel variation; they also use the within-panel variation. The former (pooled) model weighs the within and between variances equally while the latter (random-effects) model applies a weighting matrix that is "optimal" in a certain sense. If you want the to use the between-panel variation only, you need

xtreg ... , be

Third, there is no reason to believe that the within-panel and between-panel variation (or a weighted combination of the two) should result in the same estimate. For example, people who do a lot of sports have a lower risk of suffering a heart attack (between variation). However, the risk of suffering a heart attack is much higher while doing sports (within variation).

[Janet Lewis]

Thank you Daniel. I highly appreciate your comments and clarifications. This makes a lot more sense now!

[Carlo Lazzaro]

Janet:
there are some of tricks that you can apply to make your code more efficient:

 
 xtreg satisfaction number_children c.age##c.age i.Married i.Education own_health_cond partner_health_cond i.wave, fe cluster(personid)

As far as categorical variables are concerned, it is always a good habit to prefix them with -i.- even if you have two levels only. This way you do not forget to add -i-. when your categorical variables have >2 levels.
As far as continuous variable are concerned, creating interactions via -fvvarlist- notation allows you to exploit the wonderful capabilities of -margins- and -marginsplot-.

[Janet Lewis]

Thanks Carlo! I am a PhD student and only just started using Stata. Though in this time I have heard of some 'tricks' through colleagues. For instance I would literally make make year dummies which was time consuming as opposed to i.year until a workmate showed me this. I always thought it was just a matter of practice etc. Though do you have any suggestions for readings on such code? I would really like to become more proficient and coding in Stata. Thanks!

[Carlo Lazzaro]

Janet:
following William Lisowski 's wise reminder, the first sources for Stata learning are the Stata .pdf manual (along with entry-specific references) and, I would add, the help files of the Stata built-in command and, especially (as they are not documented in Stata .pdf manual), of the community-contributed modules.
Stata forum is also an endless source of problems, fixes, methods and tricks (this is the main reason why I decided to subscribe when I started using Stata. It is also worth highlighting that this forum is oftentimes really helpful, contributors are approachable and I've often wonder whether, in the real life, academic teachers/mentors/professors would be as helpful in supporting their students, junior assistants and so on. Last but not least, British/American English mother-tongue contributors try their best to undestand what people who, like me, were grown up speaking a different language, are asking, without skipping queries written (I'm obviously speaking for myself only) in a far from Oxonian English. In this respect, Nick Cox wrote me some years ago: "Dont worry, I studied at Cambridge!" ).
Stata bookstore includes many Stata press textbooks that are tailored to different research fields.
Obviously, practising with Stata databases that come with your copy of Stata and with Stata commands is a valuable way to learn Stata.

[William Lisowski]

Janet:

Since Carlo Lazzaro has called my attention to your question, I will post the advice to which he refers, and which I give whenever I read a post in which a member identifies themself as new to Stata.

I'm sympathetic to you as a new user of Stata - it's a lot to absorb. I'd like to encourage you to take a step back from your immediate tasks and do some reading. As if you have the time to take on even more reading, thinking back to when I was in your position. But still ...

When I began using Stata in a serious way, I started, as have others here, by reading my way through the Getting Started with Stata manual relevant to my setup. Chapter 18 then gives suggested further reading, much of which is in the Stata User's Guide, and I worked my way through much of that reading as well. There are a lot of examples to copy and paste into Stata's do-file editor to run yourself, and better yet, to experiment with changing the options to see how the results change.

All of these manuals are included as PDFs in the Stata installation and are accessible from within Stata - for example, through the PDF Documentation section of Stata's Help menu. The objective in doing the reading was not so much to master Stata as to be sure I'd become familiar with a wide variety of important basic techniques, so that when the time came that I needed them, I might recall their existence, if not the full syntax, and know how to find out more about them in the help files and PDF manuals.

Stata supplies exceptionally good documentation that amply repays the time spent studying it - there's just a lot of it. The path I followed surfaces the things you need to know to get started in a hurry and to work effectively.

[Nick Cox]

I would give very similar advice. I started Stata when it was much smaller, there was just one book available (Larry Hamilton's Statistics with Stata) and then there were the manuals and the help files. No Statalist or StataCorp website existed at that time.

So, nostalgia's not what it used to be, but somewhat like William, I just kept reading especially the early chapters of [U] again and again until more and more was familiar.

Teaching and learning are a still a puzzle to me. In some ways it is very inefficient to learn what you don't need and I hear of people who get where they want to be solely by using the Stata menu and never mastering the command language, which is my attitude too to much of the software I use. Precisely the opposite can also be true, namely it isn't just a matter of fumbling your way towards syntax that does what you want: you need to understand it too!

[Jeff Wooldridge]

Generally, there is nothing wrong with applying different estimation methods to the same model. When I teach this material -- as I just did the past two weeks -- I start with pooled OLS and discuss the assumptions under which it is consistent. With small T, large N, cluster-robust inference is easy. Then random effects. Then fixed effects. Then first differencing. Each of these is a valid estimator applied to the same model. If the unobserved heterogeneity is uncorrelated with the covariates then pooled OLS is consistent, and we can easily correct the standard errors. The problem with pooled OLS is not that it ignores the panel structure. It's that the omitted heterogeneity is assumed to be uncorrelated with the covariates, and this is usually a questionable assumption. RE makes the same assumption but, due to the implicit GLS transformation, it actually removes a fraction -- usually pretty large -- of the heterogeneity. One can actually get an idea about what is going on by trying all four estimation methods. In a randomized intervention setting one can use POLS with cluster-robust standard errors. And in difference-in-differences settings, pooled OLS is numerically the same as FE. So clearly POLS can't always be bad.

If we take the original equation and add the time averages of all time-varying variables, we can estimate it by POLS or RE and we always obtain the fixed effects estimates. So, again, POLS has its uses. I often start that way to see how the RE and FE estimates change.

[Janet Lewis]

Thank you all for your responses.
Carlo Lazzaro William Lisowski Nick Cox- I truly appreciate your recommendations.

It is also worth highlighting that this forum is oftentimes really helpful, contributors are approachable and I've often wonder whether, in the real life, academic teachers/mentors/professors would be as helpful in supporting their students, junior assistants and so on.

I completely agree with you on this one Carlo. I have often come across road blocks in my short time using Stata, and the help I have gotten from this forum has been invaluable with zero judgement and a very helpful community on here.

Jeff Wooldridge thanks you for your insights regarding estimation methods. I am making my way through your book. I would love to attend your lectures one day! I

[Hamid muili]

Carlo Lazzaro , please in the above code, I would love to know the reason for using if<= 3, with the vce (cluster I'd) command in the fixed effect model.
Thanks.