Estimating asymmetrical confidence intervals for ICC using -nlcom-

[Niels Henrik Bruun]

Dear all
Out of curiosity I want to reproduce the calculations from:

cls
use https://www.stata-press.com/data/r16/judges, clear
icc rating target judge, format(%6.3f)

using a mixed regressions and possibly -nlcom-:

mixed rating, reml noheader nolog nofetable ||_all: R.target ||_all: R.judge
nlcom ///
    ( individual: exp(2*_b[lns1_1_1:_cons]) / (exp(2*_b[lns1_1_1:_cons]) + (exp(2*_b[lns1_2_1:_cons]) + exp(2*_b[lnsig_e:_cons]))) ) ///
    ( average: exp(2*_b[lns1_1_1:_cons]) / (exp(2*_b[lns1_1_1:_cons]) + (exp(2*_b[lns1_2_1:_cons]) + exp(2*_b[lnsig_e:_cons])) / 4) )

The point estimates are equal for the two methods. The same is not true for the confidence intervals.

The distribution of ICC is F and it is asymmetrical. This is clearly the problem for -nlcom-.

Without success, I've tried stepwise estimation by -nlcom- estimating the log variances in the first step with option post.
Note that I have to estimate the total variance as the sum of variances from two independent variables before (or at the same time) that I estimate ICC.
Also without success I've tried building estimation of log variances and ICC into one -nlcom- like

nlcom (log_a: ...) (log_b: ...) (ICC: exp(log_a - log_b))

I have the following questions:

1. Is there a way of tricking -nlcom- into getting more exact confidence intervals? Or is it a lost cause?
2. Are the approaches I've used with -nlcom- valid?
3. If so, how do I do it right?

Looking forward to hear from you

[Maarten Buis]

The symmetric confidence interval is due to the fact that it computes those using the delta method. This is a large sample approximation, which assumes that the sampling distribution of the thing you are computing can be approximated with a normal distribution. This is often a problem with fractions.

[Niels Henrik Bruun]

Dear Maarten
Thank you for your answer.

My hope iss that someone know a hack like making a confidence interval on the log scale and transform the limit back by exp applicable on -nlcom-.
My solution to this was trying to use -nlcom- in steps and use the hack mentioned just above.

And maybe someone know of a better transformation than the log.

And how to use -nlcom- in successive steps if that is possible.

In the end I might return to the F-distribution

[Niels Henrik Bruun]

A conclusion:
-nlcom- works fine in succesive steps when one uses _b[value_name] correctly
The transformation I was looking for was something like Fisher's transformation on rho.
It definitely does not work for small samples.
Example code is:

qui mixed rating, reml noheader nolog nofetable ||_all: R.target ||_all: R.judge
nlcom ( icc_f: atanh(exp(2*[lns1_1_1]_cons - log(exp(2*[lns1_1_1]_cons) + exp(2*[lns1_2_1]_cons) + exp(2*[lnsig_e]_cons)))) ), post
di %6.3f tanh(_b[icc_f]), %6.3f tanh(_b[icc_f] - 1.96 * _se[icc_f]), %6.3f tanh(_b[icc_f] + 1.96 * _se[icc_f])

Using se = 1 / sqrt(n-3) instead does not work, obviously given the small sample size.

[Niels Henrik Bruun]

Btw: Just found "Speaking Stata: Correlation with confidence, or Fisher’s z revisited" by Cox (2008). Excellent