Dear all,
Stata does not report the F-statistic when I cluster the standard errors at the firm level. I am aware that the missing F-statistic is discussed in previous forums, however, these do not provide the answers that I am looking for. For my master thesis, I am trying to estimate the following model :
Xtreg Y $control_variables $blockholderFE i.year, fe i(firmID) vce(cluster firmID)
Where Y is an accounting variable (e.g., investment, leverage), and where $blockholderFE are 1185 dummy variables that are equal to one when a large shareholder is present in firm j in year t. I have an unbalanced panel-dataset with 1900 firms, and 21,000 firm-year observations over the years 2003-2019. I restrict large shareholders to be present in two or more firms. For those who are interested, the model is based on “Large shareholders and Corporate policies” by Cronqvist and Fahlenbrach (2009)
I am not interested in the F-statistic of the complete model, but I need to test whether the 1185 dummy variables are statistically different from zero. The command test $blockholderFE does report the F-statistics, however, the F-statistic is inflated (F-statistic > 400). In contrast, the F-statistic without vce(cluster) has a value of less than 3. I have analyzed the differences between the standard errors and robust standard errors and find that including vce(cluster) reduces the standard errors by 80% on average. I feel like the standard errors are unreliable, is that correct? Can I use the following as a reason for not clustering the standard errors at the firm level?
According to Austin Nichols and Mark Schaffer “In a fixed-effect model, where there are a large number of parameters, this often means that test of overall model significance is feasible. However, testing fewer than M linear constraints is perfectly feasible in these models, though when fixed effects and clustering are specified at the same level, tests that involve the fixed effects themselves are inadvisable (the standard errors on fixed effects are likely to be substantially underestimated, though this will not affect the other variance estimates in general” https://www.stata.com/meeting/13uk/nichols_crse.pdf
According to help j_robustsingular, the F-statistic turns missing when 1) more predictors than clusters, and 2) the presence of singleton dummies.
I don’t think that these are apparent in my case because:
1) I have more clusters (1900), than parameters (<1200)
2) I restrict large shareholders to be present in more than two firms. Hence, each dummy variable is non-zero in more than 1 observation, and the dummy variable is non-zero in multiple clusters.
I found that the F-statistic goes missing when different dummy variables are present in the same firms. (using the xtreg command)
Example:
When I ran xtreg and then the dependent variable, a set of control variables, dummy1-dummy3, firm and year fixed effects, and clustered standard errors at the firm level, Stata does report the F-statistic. However, when I add the fourth dummy, the F-statistic goes missing. For simplicity, I have shown 4 variables, but the same holds with more than 50 dummy variables. Can someone explain why this is happening?
I want to include vce(cluster) because xttest3 shows that there is heteroscedasticity. Also, xtserial suggests that there is serial correlation in a model without firm, and year fixed effects. For my analyses, I work with a model that does not cluster the standard errors, because clustering seems to underestimate the standard errors. In a model with a set of control variables, 1185 dummy variables, firm and year fixed effects, I find that the blockholder fixed effects (1185 dummy variables) are statistically different from zero (p-value <0.001). I would like to rule out that not accounting for serial correlation and heteroscedasticity results in a type 1 error. (concluding that the blockholder fixed effects are jointly significant, while actually, they are not)
Basically, I have the following questions:
(1) Clustering the standard error seems to result in underestimated standard errors. Is it therefore better not to cluster the standard errors? What would be the reasoning?
(2) Why does Stata not report the F-statistic when dummy4 is included in the example above?
(3) Are my concerns for having a type 1 error, by not accounting for heteroscedasticity and serial correlation, well-founded? If so, is there a way to mitigate this issue?
Hopefully, someone can help me.
Thanks in advance!
Corné
Stata does not report the F-statistic when I cluster the standard errors at the firm level. I am aware that the missing F-statistic is discussed in previous forums, however, these do not provide the answers that I am looking for. For my master thesis, I am trying to estimate the following model :
Xtreg Y $control_variables $blockholderFE i.year, fe i(firmID) vce(cluster firmID)
Where Y is an accounting variable (e.g., investment, leverage), and where $blockholderFE are 1185 dummy variables that are equal to one when a large shareholder is present in firm j in year t. I have an unbalanced panel-dataset with 1900 firms, and 21,000 firm-year observations over the years 2003-2019. I restrict large shareholders to be present in two or more firms. For those who are interested, the model is based on “Large shareholders and Corporate policies” by Cronqvist and Fahlenbrach (2009)
I am not interested in the F-statistic of the complete model, but I need to test whether the 1185 dummy variables are statistically different from zero. The command test $blockholderFE does report the F-statistics, however, the F-statistic is inflated (F-statistic > 400). In contrast, the F-statistic without vce(cluster) has a value of less than 3. I have analyzed the differences between the standard errors and robust standard errors and find that including vce(cluster) reduces the standard errors by 80% on average. I feel like the standard errors are unreliable, is that correct? Can I use the following as a reason for not clustering the standard errors at the firm level?
According to Austin Nichols and Mark Schaffer “In a fixed-effect model, where there are a large number of parameters, this often means that test of overall model significance is feasible. However, testing fewer than M linear constraints is perfectly feasible in these models, though when fixed effects and clustering are specified at the same level, tests that involve the fixed effects themselves are inadvisable (the standard errors on fixed effects are likely to be substantially underestimated, though this will not affect the other variance estimates in general” https://www.stata.com/meeting/13uk/nichols_crse.pdf
According to help j_robustsingular, the F-statistic turns missing when 1) more predictors than clusters, and 2) the presence of singleton dummies.
I don’t think that these are apparent in my case because:
1) I have more clusters (1900), than parameters (<1200)
2) I restrict large shareholders to be present in more than two firms. Hence, each dummy variable is non-zero in more than 1 observation, and the dummy variable is non-zero in multiple clusters.
I found that the F-statistic goes missing when different dummy variables are present in the same firms. (using the xtreg command)
Example:
| FirmID | Year | Dummy1 | Dummy2 | Dummy3 | Dummy4 |
| 1 | t | 1 | 1 | 0 | 1 |
| 1 | t | 1 | 1 | 1 | 1 |
| 1 | t | 1 | 0 | 1 | 0 |
| 1 | t | 0 | 0 | 0 | 0 |
| 1 | t | 0 | 0 | 0 | 0 |
| 2 | t | 1 | 1 | 0 | 0 |
| 2 | t | 1 | 1 | 0 | 0 |
| 2 | t | 0 | 0 | 1 | 0 |
| 2 | t | 0 | 0 | 1 | 0 |
| 2 | t | 0 | 0 | 0 | 0 |
| 3 | t | 0 | 1 | 0 | 0 |
| 3 | t | 0 | 1 | 1 | 0 |
| 3 | t | 0 | 0 | 1 | 0 |
| 3 | t | 0 | 0 | 0 | 0 |
| 3 | t | 0 | 0 | 0 | 0 |
| 4 | t | 0 | 0 | 1 | 0 |
| 4 | t | 0 | 0 | 1 | 1 |
| 4 | t | 0 | 0 | 1 | 1 |
| 4 | t | 0 | 0 | 0 | 1 |
| 4 | t | 0 | 0 | 0 | 0 |
I want to include vce(cluster) because xttest3 shows that there is heteroscedasticity. Also, xtserial suggests that there is serial correlation in a model without firm, and year fixed effects. For my analyses, I work with a model that does not cluster the standard errors, because clustering seems to underestimate the standard errors. In a model with a set of control variables, 1185 dummy variables, firm and year fixed effects, I find that the blockholder fixed effects (1185 dummy variables) are statistically different from zero (p-value <0.001). I would like to rule out that not accounting for serial correlation and heteroscedasticity results in a type 1 error. (concluding that the blockholder fixed effects are jointly significant, while actually, they are not)
Basically, I have the following questions:
(1) Clustering the standard error seems to result in underestimated standard errors. Is it therefore better not to cluster the standard errors? What would be the reasoning?
(2) Why does Stata not report the F-statistic when dummy4 is included in the example above?
(3) Are my concerns for having a type 1 error, by not accounting for heteroscedasticity and serial correlation, well-founded? If so, is there a way to mitigate this issue?
Hopefully, someone can help me.
Thanks in advance!
Corné
Comment