[Clyde Schechter]

Again, we have to distinguish two meanings of "clustering."

In terms of specifying a level in a random- or mixed-effects model, you generally wouldn't do it with only 5-10 entities at that level because a random sample of only 5-10 from a normal distribution gives extremely imprecise estimates of its variance. So the results you get from including random effects at that level are just not useful.

In terms of specifying clustering for cluster robust variance estimators, simulations have shown that the standard errors provided by the cluster robust vce are actually worse than the ordinary standard errors when the number of clusters is that small. VCE has good large sample (# of clusters) properties, but works poorly in small samples. Unfortunately, as far as I know, this is backed up only by limited simulation studies and, for that reason, there is no rigorous basis for knowing how few is too few and how many is enough under what circumstances.

[Krissy Philips]

Sorry yes I mean clustering of se when I always use the word clustering. That's very useful though, thank you, you've been extremely helpful!

Am I right in assuming that clustering the standard errors at the company level is the same as using robust standard errors? So by using ,robust - I am infact killing two birds with one stone?

[Clyde Schechter]

In -xtreg, fe-, specifying vce(robust) and vce(cluster varname) are the same thing. But, as far as I know, that is only true for -xtreg, fe-.

But if you will be using -xtreg, re-, and for that -vce(robust)- does not take into account the clustering. You need to specify -vce(cluster company)- to get the cluster robust standard error here. The cluster robust standard error, in addition to accounting for clustering of observations, is also robust to heteroscedasticity and some mis-specification of the model. Thus you do "kill two birds with one stone" when you use -vce(cluster company)-.

[Krissy Philips]

In -xtreg, fe-, specifying vce(robust) and vce(cluster varname) are the same thing. But, as far as I know, that is only true for -xtreg, fe-.

But if you will be using -xtreg, re-, and for that -vce(robust)- does not take into account the clustering. You need to specify -vce(cluster company)- to get the cluster robust standard error here. The cluster robust standard error, in addition to accounting for clustering of observations, is also robust to heteroscedasticity and some mis-specification of the model. Thus you do "kill two birds with one stone" when you use -vce(cluster company)-.

Do you mean when you use "-vce(robust)" you kill two birds? Because you said the cluster robust se accounts for clustering of observations and robust to heteroscedasticity, so surely it is the robust option that is better better?

[Clyde Schechter]

1. In general, the cluster robust covariance estimator is robust to both heteroscedasticity and adjusts standard errors to reflect the clustering of observations within whatever clustering variable is specified. It "kills two birds." The simple robust estimator does not adjust standard errors to reflect clustering.

2. However, in the context of -xtreg, fe- (and, as far as I know, only in this context), when you specify -vce(robust)-, Stata substitutes -vce(cluster panelvar)- instead. In Stata, you cannot get the simple robust estimator with -xtreg, fe-.

[Krissy Philips]

Again, we have to distinguish two meanings of "clustering."

...simulations have shown that the standard errors provided by the cluster robust vce are actually worse than the ordinary standard errors when the number of clusters is that small. VCE has good large sample (# of clusters) properties, but works poorly in small samples. Unfortunately, as far as I know, this is backed up only by limited simulation studies and, for that reason, there is no rigorous basis for knowing how few is too few and how many is enough under what circumstances.

Would you be able to kindly provide the names/links to such simulation studies please Clyde?

Many thanks

[Clyde Schechter]

Take a look at http://cameron.econ.ucdavis.edu/rese...5_February.pdf. This is not one of those simulation studies, but it is a very complete discussion of all aspects of cluster robust VCE and it includes references on this specific topic. Some of the paper is very "mathy", but if that isn't to your taste, you can skip over those parts and read the narrative--it's a really complete explanation of the topic in all its aspects.

[Jonas Haas]

But if you use company as your panel variable in -xtset-, then your model will treat observations within firms as independent--which they almost certainly are not.
Do remember, though, assuming that each firm stays in the same country during the time it is included in your data, the use of firm level country effects will automatically adjust for any country-level time-invariant effects without your having do do anything special to make that happen. What it won't do is let you estimate those country-level effects (because they will be colinear with the firm-fixed effects). It sounds like you have a multi-level model:monthly observations nested in firms nested in countries. So if estimating country-level effects are important to your goals, you need a multi-level model, or perhaps you would estimate the country level effects separately using -xtset country-, -xtreg, be-.

Dear Clyde,
this is an old thread but I have a similar doubt as the OP. I'm running a conditional logit regression on panel data with person-year unit of analysis. Individuals are nested within households. Fixed effects logit (and sometimes linear probability) is the standard method in this literature which is why I would prefer it over random (mixed) effects models.
My intuition was to include household fixed effects and I got the results I would expect (positive and negative signs at the coefficients in line with literature). After reading your post though, I included individual fixed effects at the individual level and some signs changed from negative to positive. Now I'm rather confused. I guess my identification strategy was wrong. Could you maybe explain a bit further what you mean with "your model will treat observations within [households] as independent" and why applying household fixed effects instead of fixed effects at the unit id might have biased the results?

[Clyde Schechter]

A limitation of fixed-effects models is that they only really work with two-level data. When you have three-level data and you use fixed-effects models you are necessarily putting the data to sleep in a Procrustean bed. Something will go wrong, no matter how you do it.

In trying to understand the implications of the different ways of doing it, a key thing to bear in mind is that the same variable can mean different things depending on what the fixed effect is. Fixed-effects models estimate the effects of variables within the fixed-effect panel. So if you use individual as the fixed effect in your model, all of your estimated effects refer to the effects of changes in the variable within the same individual over time. If you use the household as the fixed effect, then your estimated effects refer to the changes in the variable across different people within the same household. It is entirely possible for those effects to be very different, even of opposite signs. It is unlikely that both of these effects are what you want for your research goals. (It may be the case that neither is.) If one of them is the appropriate thing to estimate for your research goals, then choose the fixed effect accordingly.

As you are working in a field where fixed-effect modeling is strongly privileged, I assume you have already been taught the limitations of random-effects models. Possibly you have been taught to avoid them like the plague--that is unfortunately what is widely taught in some disciplines. The fact remains that for multi-level model of this type no model is perfect. There are no right answers, only more and less wrong ones. In epidemiology we use random-effects models much more liberally. In part, that is because we often deal with randomized trial data, which overcomes the major limitations of random effects models. But even with non-randomized data, in my field we will generally prefer to be sure we estimate the right effect (within vs between) for our goals, and do our best to meet the independence of residuals and predictors assumptions by adjusting for as many confounders as we can.

[Jonas Haas]

A limitation of fixed-effects models is that they only really work with two-level data. When you have three-level data and you use fixed-effects models you are necessarily putting the data to sleep in a Procrustean bed. Something will go wrong, no matter how you do it.

In trying to understand the implications of the different ways of doing it, a key thing to bear in mind is that the same variable can mean different things depending on what the fixed effect is. Fixed-effects models estimate the effects of variables within the fixed-effect panel. So if you use individual as the fixed effect in your model, all of your estimated effects refer to the effects of changes in the variable within the same individual over time. If you use the household as the fixed effect, then your estimated effects refer to the changes in the variable across different people within the same household. It is entirely possible for those effects to be very different, even of opposite signs. It is unlikely that both of these effects are what you want for your research goals. (It may be the case that neither is.) If one of them is the appropriate thing to estimate for your research goals, then choose the fixed effect accordingly.

As you are working in a field where fixed-effect modeling is strongly privileged, I assume you have already been taught the limitations of random-effects models. Possibly you have been taught to avoid them like the plague--that is unfortunately what is widely taught in some disciplines. The fact remains that for multi-level model of this type no model is perfect. There are no right answers, only more and less wrong ones. In epidemiology we use random-effects models much more liberally. In part, that is because we often deal with randomized trial data, which overcomes the major limitations of random effects models. But even with non-randomized data, in my field we will generally prefer to be sure we estimate the right effect (within vs between) for our goals, and do our best to meet the independence of residuals and predictors assumptions by adjusting for as many confounders as we can.

Thank you for the clarifications. Yes indeed, fixed effects seem to be expected in econometrics, at least in the subfield I'm working currently working in. I read maybe one or two papers in my research field that explored mixed models, and I'm interested in learning more about them in the future, however there seems to be a strong consensus in my field that the random effects assumption is violated. So presently working out a new method and identification strategy, unfortunately does not fit in the time frame.

Considering your remarks I am quite certain that applying household fixed effects is the wrong strategy. I am definitely interested in the 'within' effect and I assumed household fixed effects would give me those within effects for households over time. I will go forward estimating individual (and year) fixed effects.

[Jonas Haas]

A limitation of fixed-effects models is that they only really work with two-level data. When you have three-level data and you use fixed-effects models you are necessarily putting the data to sleep in a Procrustean bed. Something will go wrong, no matter how you do it.

In trying to understand the implications of the different ways of doing it, a key thing to bear in mind is that the same variable can mean different things depending on what the fixed effect is. Fixed-effects models estimate the effects of variables within the fixed-effect panel. So if you use individual as the fixed effect in your model, all of your estimated effects refer to the effects of changes in the variable within the same individual over time. If you use the household as the fixed effect, then your estimated effects refer to the changes in the variable across different people within the same household. It is entirely possible for those effects to be very different, even of opposite signs. It is unlikely that both of these effects are what you want for your research goals. (It may be the case that neither is.) If one of them is the appropriate thing to estimate for your research goals, then choose the fixed effect accordingly.

As you are working in a field where fixed-effect modeling is strongly privileged, I assume you have already been taught the limitations of random-effects models. Possibly you have been taught to avoid them like the plague--that is unfortunately what is widely taught in some disciplines. The fact remains that for multi-level model of this type no model is perfect. There are no right answers, only more and less wrong ones. In epidemiology we use random-effects models much more liberally. In part, that is because we often deal with randomized trial data, which overcomes the major limitations of random effects models. But even with non-randomized data, in my field we will generally prefer to be sure we estimate the right effect (within vs between) for our goals, and do our best to meet the independence of residuals and predictors assumptions by adjusting for as many confounders as we can.

Yesterday after reading your comment I was ready to lay this issue to rest until I read the following paper by Gray et al (2013) (https://doi.org/10.1007/s13524-012-0192-y). They seem to be using regional and time fixed effects with a panel that has person-years as the unit of analysis. Their argument is that the inclusion of regional and time fixed effects "accounts for national- scale time-varying factors and for time-invariant factors at the scale of the study area as long as the effects are linear. Thus, the coefficients can be interpreted as comparing two individuals in the same study area who are exposed to the same changing national context over time."

Would you agree with this interpretation? How does this relate to the assumption that individuals (and the households they are nested in) are independent from each other?

[Clyde Schechter]

I probably agree with it. It is consistent with what I said earlier: "If you use the household as the fixed effect, then your estimated effects refer to the changes in the variable across different people within the same household." because in their study the region plays the role that household does in your study and their household plays the role of the individual in your study.

My hesitation to fully endorse it comes from other considerations. They used a complex survey sampling design, and I don't know enough about the implications of that to say whether it complicates or nullifies this aspect of interpreting the results. Also, they are using unconditional (multinomial) logistic regression, but it appears they have only a small number of observations per household, which may be problematic in its own right. These issues deal with aspects of statistics that I have only limited familiarity with, enough that I am aware of potential difficulties, but not enough to know whether that potential is realized.