Power Analysis [power] in Context of Regression Analysis and Robust Standard Errors

[Olaf Hotte]

Hi,

Based on the results of a regression analysis, I would like to perform a power analysis that accounts for robust standard errors.
To my knowledge, robust standard errors are usually greater than "non-robust" standard errors. This is why I would expect lower "power".

Here is an example:


"Non-robust" standard errors

sysuse auto

regress mpg price headroom weight displacement gear_ratio, beta


      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(5, 68)        =     26.18
       Model |  1608.01499         5  321.602998   Prob > F        =    0.0000
    Residual |  835.444469        68  12.2859481   R-squared       =    0.6581
-------------+----------------------------------   Adj R-squared   =    0.6329
       Total |  2443.45946        73  33.4720474   Root MSE        =    3.5051

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|                     Beta
-------------+----------------------------------------------------------------
       price |  -.0001408   .0001736    -0.81   0.420                -.0717795
   headroom |  -.3553828   .5714143    -0.62   0.536                -.0519664
      weight |  -.0061447   .0012576    -4.89   0.000                 -.825451
displacement |   .0099741   .0119302     0.84   0.406                 .1583254
  gear_ratio |   .9957116   1.659936     0.60   0.551                 .0785291
       _cons |   36.81308   6.773891     5.43   0.000                        .
------------------------------------------------------------------------------

By using the option "beta" I also get the standardized coefficients, so I can easily calculate the power. As example I use "headroom".

power onemean 0 -.0519664, n(74) sd(1)

Estimated power for a one-sample mean test
t test
Ho: m = m0  versus  Ha: m != m0

Study parameters:

        alpha =    0.0500
            N =        74
        delta =   -0.0520
           m0 =    0.0000
           ma =   -0.0520
           sd =    1.0000

Estimated power:

        power =    0.0726

With newer versions of STATA this should work as well:

power oneslope 0 -.0519664, sdx(1) sdy(1)

Robust standard errors

regress mpg price headroom weight displacement gear_ratio, beta robust


Linear regression                               Number of obs     =         74
                                                F(5, 68)          =      27.11
                                                Prob > F          =     0.0000
                                                R-squared         =     0.6581
                                                Root MSE          =     3.5051

------------------------------------------------------------------------------
             |               Robust
         mpg |      Coef.   Std. Err.      t    P>|t|                     Beta
-------------+----------------------------------------------------------------
       price |  -.0001408   .0001909    -0.74   0.463                -.0717795
   headroom |  -.3553828    .445874    -0.80   0.428                -.0519664
      weight |  -.0061447   .0010529    -5.84   0.000                 -.825451
displacement |   .0099741   .0090296     1.10   0.273                 .1583254
  gear_ratio |   .9957116   1.787227     0.56   0.579                 .0785291
       _cons |   36.81308   6.579494     5.60   0.000                        .
------------------------------------------------------------------------------

I receive the same standardized beta coefficients as with "non-robust" standard errors. And as a result the same power analysis.

How can I correct for robust standard errors?

[Carlo Lazzaro]

Olaf:
the issue is that standard errors have no role in the standardization process, as you can see from: https://www.ssc.wisc.edu/~hemken/Sta...ts%20Right.pdf.

[Olaf Hotte]

Hello Carlo,
Thank you very much! Consequently, my approach is not valid regarding "robust" power analysis.
Is there any other approach I could use?
So far, I haven't found anything that depart from these classical standards, either because of using nonstandard estimators or because of other wrinkles with the data generation or selection process.

[Carlo Lazzaro]

Olaf:
can't you circumvent the issue and consider -power rsquared- instead?

[Olaf Hotte]

No, unfortunately not. The value of the R2 is unaffected in regressions with robust standard errors. So, I face the same problem as before.

[Olaf Hotte]

I guess the only way to handle this problem is to do a simulation... Any ideas how to perform this in STATA with a given dataset and regression coefficients?

[Carlo Lazzaro]

Olaf:
see: https://blog.stata.com/2019/01/10/ca...-1-the-basics/
https://blog.stata.com/2019/01/29/ca...n-using-power/

[Olaf Hotte]

Great, thank you so much!
I worked this through. Although simulation was not the right approach for me, bootstraping is!