Test of whether country variance is statistically significant

[Anders Engberg Petersen]

I'm looking at social origin effect on occupational attainment (in terms of ISEI) in three different countries (Sweden, Germany and the UK). I have run three individual regressions and found variations in the effect but want to test whether this variation is statistically significant.

A study of the same issue in 12 European countries - by Christina Iannelli (2002) - uses a regression model which include all the countries as dummy variables. Here the effect of the country variables are statistically significant and the study concludes that there is a 'substantial' difference between the countries but is this a valid way to do it? I mean, does this test whether the effect of social origin varies significantly between the countries? Or should I include an interaction term between country and social origin?

Moreover, a similar study (Bernardi and Ballarino 2016) compare the same effect in 14 European countries (done by different scholars and datasets in each country) and compare the results without testing whether the country variance is statistically significant. The study emphasizes the 'common patterns' more than the differences, but can one compare country differences without testing whether these are significant?

[Carlo Lazzaro]

Anders:
1) you should perform an unique regression including -i.country- (see -fvvarlist- for categorical variable and interactions notation). as a predictor. Please note that, when you wrote that Iannelli (2002) (as per FAQ full reference please. Thanks) included all the countries as dummy variables, one of them had actually been left out to shelter the regression model from dummy trap (see: https://en.wikipedia.org/wiki/Dummy_...le_(statistics)).
2) you can obviously test whether an interaction exists between -i.country- and -i.social_origin- in the regressio model mentioned at 1).

[Anders Engberg Petersen]

Thanks a lot, Carlo, for your quick reply. Very helpful!

Do you have any comments on why other comparative studies about social origin effects on occupational attainment would leave out a test of whether the country variance is statistically significant?


-And thank you for the reminder about references. The full references for the mentioned studies are:

Iannelli, Cristina. 2002. Parental Education and Young People’s Educational and Labour Market Outcomes: A Comparison across Europe. 45. MZES.

Bernardi, Fabrizio and Gabrielle Ballarino, eds. 2016. Education, Occupation, and Social Origin: A Comparative Analysis of the Transmission of Socio-Economic Inequalities. Cheltenham, UK: Edward Elgar Publishing.

[Carlo Lazzaro]

Anders:
- I'm not sure about what you mean by "country variance". Do you mean the effect on variation on the occupational attainment (when adjusted for the remaining predictors) due to -i.country-? Or else?
That said, you can use -testparm- as a post estimation test to check whether -i-country- actually reaches statistical significance:

testparm(i.country)

Thanks for providing full references.
As an aside, since I've had the pleasant chance to meet him about 10 years ago as he organized an interesting workshop at the School of Political Science, University of Milan (where I was happy to meet Maarten Buis , who gave, in turn, an even more interesting speech concerning his PhD dissertation), the given name of the second editor of the second reference you quoted is Gabriele (one el only).

[Anders Engberg Petersen]

I can see why as it is not formulated very clearly. Let me try again. By country variation, I mean the difference in the effect of social origin (parental occupation) on occupational attainment there is between the European countries (measured by the coefficients from OLS regression).

Both Iannelli and Bernardi & Ballarino use OLS regression (one model estimated per country) to estimate this effect in various countries and compare the results without testing whether the different effect found in different countries are statistically significant. This just struck me as rather odd.

thank you for both the post estimation command and the tip about Gabriele

Best regards

Anders

[Carlo Lazzaro]

Anders:
thanks for providing further clarifications.
You should probably looking for something along the following lines (I assume that you have a cross-sectional dataset and your dependent variable is polytomous):

mlogit occupational_attainment i.country##i.social_origin <other controls>

If your dependent variable were continuous, the abovementioned code becomes:

regress occupational_attainment i.country##i.social_origin <other controls>

[Maarten Buis]

Anders measures occupational attainment with ISEI, which is intended to be used as a continuous variable. So regress is fine.

As to what Gabriele and Fabrizio did in their book is collect studies from country experts. So there is not in a single dataset or model and thus you cannot use interactions to explicitly test the difference. With country comparisons there is always a trade-off between using a model and data that is most appropriate for each specific country and making the models and data similar enough across countries to easily compare them. The strategy Gabrielle and Frabizio chose was to first meet with the country experts and agree as much as possible on the common strategy as to the question that is to be answered for each country, the kind of data that is to be used and the kind of model that is to be used. After that, they left the individual authors enough space, within these bounds, to do something that is appropriate for their country. I think that that is a valid strategy. It means you cannot do explicit statistical tests, but the datasets used are typically big enough, and the differences found are typically big enough, that statistical testing is not going to add much anyhow.

[Anders Engberg Petersen]

Carlo: Thanks a lot. I will test the statistical significance with regress and an interaction term.

Maarten: Thank you for clarifying the approach chosen by Barnardi and Ballarina. It makes good sense when you explain it like that and I agree with you, that it is a valid strategy. It is also worth mentioning that their book is really great.