Incorrect Z-score calculation in Stata?

[Irene Margaret]

Hi,

I have a rather trivial question but it's been bugging me for days. So I'd appreaciate some tehnical help from this forum.

I'm working on a panel dataset comprising 3,168 firms from 47 countries for 7 years (2012-2108).
Using this dataset, I want to capture the relative positioning of a country based on an institutional characteristic (say X), so for this purpose I calculate the Z-score of institutional variable X.
The idea is that countries with positive and larger Z-score (closer to 1) would be the representative of the institutional characteristics.
(For interested readers, what I do is similar to the standardized institutional variable in Gupta et al. (2020)) https://sms.onlinelibrary.wiley.com/....1002/smj.3204

I calculate the Z-score using the following code

 bys year: egen z_institutionX = std(institutionX)

I also do it manually by calculating the mean and standard deviation of variable X for each year

 bys year: egen mean_institutionX = mean(institutionX)

 bys year: egen sd_institutionX = sd(institutionX)

 bys year: gen manualz_institutionX = (institutionX - mean_institutionX)/sd_institutionX

To my surprise, the mean and standard deviation produced by Stata is substantially larger, and thereby the Z-score would be incorrect. Here I give one example for year 2013.
N is the number of firm observations in a country. SD for individual country is zero because the value of institutional variable X is the same for every firms in the same country.
With manual calculation, the mean should be 0.708317 and standard deviation 0.120099.
I don't think the difference is mainly driven by rounds up. Would appreaciate any enlighment on this issue. Thanks!

 
hq
N
Mean
SD




Australia
138
.8116238
0

Austria
12
.5283403
0

Belgium
14
.4955547
0

Denmark
18
.5816855
0

Finland
22
.6569287
0

Germany
63
.6950699
0

Japan
314
.7472216
0

Netherlands
28
.7077156
0

Norway
12
.7062369
0

Sweden
28
.7839157
0

Switzerland
54
.860143
0

United Kingdom
178
.7391919
0

United States
500
.8944967
0




Total
1381
.7995181
.0882503

[Andrew Musau]

Stata's calculation is a weighted mean, i.e., it takes into account that the sample sizes differ between countries. In your case, you are assigning equal weight to each country.

* Example generated by -dataex-. For more info, type help dataex
clear
input str14 hq int n float mean
"Australia"      138 .8116238
"Austria"         12 .5283403
"Belgium"         14 .4955547
"Denmark"         18 .5816855
"Finland"         22 .6569287
"Germany"         63 .6950699
"Japan"          314 .7472216
"Netherlands"     28 .7077156
"Norway"          12 .7062369
"Sweden"          28 .7839157
"Switzerland"     54  .860143
"United Kingdom" 178 .7391919
"United States"  500 .8944967
end

mean mean
mean mean [fw=n]

Res.:

. mean mean

Mean estimation                             Number of obs = 13

--------------------------------------------------------------
             |       Mean   Std. err.     [95% conf. interval]
-------------+------------------------------------------------
        mean |   .7083172   .0333094      .6357424    .7808921
--------------------------------------------------------------

. 
. mean mean [fw=n]

Mean estimation                          Number of obs = 1,381

--------------------------------------------------------------
             |       Mean   Std. err.     [95% conf. interval]
-------------+------------------------------------------------
        mean |    .799518   .0023748      .7948595    .8041766
--------------------------------------------------------------

[Irene Margaret]

Thanks Andrew! I'll run the analysis using Z-score from both weighted and unweighted mean adn see whether there is substantial differences.

[Nick Cox]

I confirm that z-scores using egen's std() directly and its mean() and sd() functions indirectly should yield identical results.

I can't follow precisely what you're doing otherwise. Is what you tabulate means and SDs of the original variable or means and SDs of z scores?. If the latter I would expect some negative means.

My only guess is that your puzzle boils down to standardizing by year but having quite different numbers of observations for each country.

z scores have mean 0 and SD 1 by default but I can't see any reason to regard values close to 1 as being representative in any sense. (I haven't read your link.)

EDIT: Written before I read Andrew Musau's post.

[Irene Margaret]

Thanks Nick. Besides Andrew's reply, I also have looked at earlier discussions on egen function wmean() to assess how different weights could affect my Z-score calculations.
https://www.statalist.org/forums/for...-harmonic-mean