Confidence interval of variable derived from linear regression was different from by syntax -ci means-

[Tom Hsiung]

The response variable is inr and the independent variable is day. Day has values from 1 to 14, discrete quantitative variable. I noticed that by two different methods to compute the confidence interval for each independent variable value (1 to 14) resulted in different standard errors and confidence interval, as shown below.

. regress inr i.day

      Source |       SS           df       MS      Number of obs   =       690
-------------+----------------------------------   F(13, 676)      =     18.76
       Model |  37.8816699        13   2.9139746   Prob > F        =    0.0000
    Residual |  105.006267       676  .155334715   R-squared       =    0.2651
-------------+----------------------------------   Adj R-squared   =    0.2510
       Total |  142.887937       689  .207384524   Root MSE        =    .39413

------------------------------------------------------------------------------
         inr |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         day |
          2  |   .2573988   .0679332     3.79   0.000     .1240134    .3907842
          3  |   .5258266   .0676578     7.77   0.000      .392982    .6586713
          4  |   .6791819   .0668741    10.16   0.000     .5478759    .8104878
          5  |   .7021678   .0666261    10.54   0.000     .5713488    .8329868
          6  |   .6818289   .0668741    10.20   0.000      .550523    .8131349
          7  |   .6358363   .0679332     9.36   0.000     .5024509    .7692217
          8  |    .654484    .070796     9.24   0.000     .5154774    .7934906
          9  |   .6719555   .0761594     8.82   0.000     .5224181    .8214928
         10  |   .6234164   .0785541     7.94   0.000     .4691769    .7776559
         11  |   .6855143   .0830357     8.26   0.000     .5224754    .8485532
         12  |   .7540113   .0916581     8.23   0.000     .5740425      .93398
         13  |   .9027113   .1090699     8.28   0.000     .6885549    1.116868
         14  |   .9362113   .1331213     7.03   0.000     .6748303    1.197592
             |
       _cons |   1.114789   .0467741    23.83   0.000     1.022949    1.206629
------------------------------------------------------------------------------

. margins i.day

Adjusted predictions                            Number of obs     =        690
Model VCE    : OLS

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         day |
          1  |   1.114789   .0467741    23.83   0.000     1.022949    1.206629
          2  |   1.372187   .0492657    27.85   0.000     1.275455     1.46892
          3  |   1.640615   .0488852    33.56   0.000      1.54463    1.736601
          4  |   1.793971   .0477947    37.53   0.000     1.700127    1.887815
          5  |   1.816957   .0474471    38.29   0.000     1.723795    1.910118
          6  |   1.796618   .0477947    37.59   0.000     1.702774    1.890462
          7  |   1.750625   .0492657    35.53   0.000     1.653893    1.847357
          8  |   1.769273   .0531438    33.29   0.000     1.664926     1.87362
          9  |   1.786744   .0601035    29.73   0.000     1.668732    1.904756
         10  |   1.738205   .0631105    27.54   0.000     1.614289    1.862121
         11  |   1.800303   .0686084    26.24   0.000     1.665592    1.935014
         12  |     1.8688   .0788251    23.71   0.000     1.714029    2.023571
         13  |     2.0175   .0985313    20.48   0.000     1.824036    2.210964
         14  |      2.051   .1246333    16.46   0.000     1.806285    2.295715
------------------------------------------------------------------------------

. sort day

. ci means inr in 1/71

    Variable |        Obs        Mean    Std. Err.       [95% Conf. Interval]
-------------+---------------------------------------------------------------
         inr |         71    1.114789    .0099977        1.094849    1.134729

.

The reasons I did this is to get a graph like below. I notice the title "Adjusted". Actually, I don't want to some adjustment in my case. It seems that the Stata has made the adjustment, which might be the cause for difference in confidence interval. There is only one independent variable -- "day", but the linear regression analysis listed 13 variables. That's the cause, right?

[William Lisowski]

There is only one independent variable -- "day", but the linear regression analysis listed 13 variables.

No, by using factor variable notation i.day you have instructed Stata to use your variable day - which takes on integer values 1, 2, ..., 14 - to create 14 indicator variables, one of which is then omitted from the regression to eliminate collinearity. As help factor variables tells us

Factor variables create indicator variables from categorical variables, interactions of indicators of categorical variables, interactions of categorical and continuous variables, and interactions of continuous variables (polynomials).

I'll leave it to someone else to explain the difference between your output from margins based on your regression model and your output from ci on a subset of your data.

Added in edit: A good way to improve your understanding of the margins command would be to read Richard Williams's expository pieces, often recommended on Statalist, links to which are given in the section headed "Interpreting results: Adjusted Predictions and Marginal effects" at

https://www3.nd.edu/~rwilliam/stats3/

[Marcos Almeida]

Please do take some time to get the core knowledge about linear regression. You could also take a look at: the reference level; factor notation; how changing the reference level would modify the coefficients; the mean for the whole sample; the mean for each level of a categoricalvariable; the ‘traditional’ formula of the CI; predictive values; margins and the delta method; the assumptions of the ‘standard’ linear regression; repeated measures and time-series.

All these topics are dutifully approached ( and freely available) in the Stata Manual, but adding extra information from a good textbook would be helpful as well. Needless to say, Richard’s renditions of - margins - (as also suggested by William) are a hit or miss.

[Tom Hsiung]

I know that by two groups t-test, the -ttest- would feedback the means and confidence intervals of group A and group B, respectively. By a simple linear regression, the -margins- also feedback the means and confidence intervals of group A and group, however, the confidence intervals by two methods are different because the standard error estimated are different. However, the means of group A and group B by two different methods are the same.

If, I added a second independent variable to the linear regression, however, the means by two method would be different.

Compare the following

. regress averagedailydoseduringtheinitial i.nonaaornonanona

      Source |       SS           df       MS      Number of obs   =       200
-------------+----------------------------------   F(1, 198)       =     21.56
       Model |  12.4834566         1  12.4834566   Prob > F        =    0.0000
    Residual |  114.624253       198  .578910369   R-squared       =    0.0982
-------------+----------------------------------   Adj R-squared   =    0.0937
       Total |   127.10771       199  .638732209   Root MSE        =    .76086

-----------------------------------------------------------------------------------
averagedailydos~l |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
1.nonaaornonanona |   .7554286   .1626791     4.64   0.000     .4346226    1.076234
            _cons |   2.581771   .0575157    44.89   0.000     2.468349    2.695193
-----------------------------------------------------------------------------------

. margins i.nonaaornonanona

Adjusted predictions                            Number of obs     =        200
Model VCE    : OLS

Expression   : Linear prediction, predict()

---------------------------------------------------------------------------------
                |            Delta-method
                |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
nonaaornonanona |
             0  |   2.581771   .0575157    44.89   0.000     2.468349    2.695193
             1  |     3.3372   .1521723    21.93   0.000     3.037114    3.637286
---------------------------------------------------------------------------------

. ttest averagedailydoseduringtheinitial, by(nonaaornonanona)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
       0 |     175    2.581771    .0542955    .7182624    2.474609    2.688934
       1 |      25      3.3372    .2035416    1.017708    2.917111    3.757289
---------+--------------------------------------------------------------------
combined |     200      2.6762    .0565125    .7992072     2.56476     2.78764
---------+--------------------------------------------------------------------
    diff |           -.7554286    .1626791               -1.076234   -.4346226
------------------------------------------------------------------------------
    diff = mean(0) - mean(1)                                      t =  -4.6437
Ho: diff = 0                                     degrees of freedom =      198

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 1.0000

. ci means averagedailydoseduringtheinitial if(nonaaornonanona==0)

    Variable |        Obs        Mean    Std. Err.       [95% Conf. Interval]
-------------+---------------------------------------------------------------
averagedai~l |        175    2.581771    .0542955        2.474609    2.688934

. ci means averagedailydoseduringtheinitial if(nonaaornonanona==1)

    Variable |        Obs        Mean    Std. Err.       [95% Conf. Interval]
-------------+---------------------------------------------------------------
averagedai~l |         25      3.3372    .2035416        2.917111    3.757289

.

and

. regress averagedailydoseduringtheinitial i.nonaaornonanona bsa

      Source |       SS           df       MS      Number of obs   =       200
-------------+----------------------------------   F(2, 197)       =     24.72
       Model |  25.5026633         2  12.7513316   Prob > F        =    0.0000
    Residual |  101.605046       197  .515761657   R-squared       =    0.2006
-------------+----------------------------------   Adj R-squared   =    0.1925
       Total |   127.10771       199  .638732209   Root MSE        =    .71817

-----------------------------------------------------------------------------------
averagedailydos~l |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
1.nonaaornonanona |    .794156   .1537436     5.17   0.000     .4909615    1.097351
              bsa |   1.364822   .2716491     5.02   0.000     .8291083    1.900535
            _cons |   .3032125   .4567537     0.66   0.508    -.5975418    1.203967
-----------------------------------------------------------------------------------

. margins i.nonaaornonanona

Predictive margins                              Number of obs     =        200
Model VCE    : OLS

Expression   : Linear prediction, predict()

---------------------------------------------------------------------------------
                |            Delta-method
                |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
nonaaornonanona |
             0  |    2.57693   .0542968    47.46   0.000     2.469853    2.684008
             1  |   3.371087   .1437914    23.44   0.000     3.087519    3.654654
---------------------------------------------------------------------------------

.

[William Lisowski]

That adding a second independent variable changes the reported means is the hint to you that margins is not reporting the means of the observations in each group, but something different - in particular, "adjusted means."

Please do spend the time to increase your understanding of the purpose of the margins command, which is not to produce plots of group means and their confidence intervals as reported by the ci command. You should find another way to create that plot.

[Tom Hsiung]

Hi, William

The issue is, now I did a multiple linear regression. I would like to post the confidence interval plots between groups in my paper. I use the -margins- and -marginsplot- command to do this. However, according to your reply, it was inappropriate?

For the simplistic, I have a response variable of Dose, and three independent variables of Gene1 CHF (congestive heart failure) and AKI (acute kidney injury). So Dose = b0 + b1 x Gene1 + b2 x CHF + b3 x AKI. The three independent variables are all dichotomic variables (0 vs 1). Now I would like to plot the confidence interval plot between groups. Is is appropriate to use the -margins- and -marginsplot-?

PS: the -ci means- command could list the confidence interval of the response variable day by day.

. ci means inratday*

    Variable |        Obs        Mean    Std. Err.       [95% Conf. Interval]
-------------+---------------------------------------------------------------
   inratday1 |        112    1.115714    .0098833         1.09613    1.135299
   inratday2 |        105    1.348667    .0256259         1.29785    1.399484
   inratday3 |        100      1.5989    .0405535        1.518433    1.679367
   inratday4 |        107    1.797664    .0480084        1.702482    1.892845
   inratday5 |        104     1.84125    .0485395        1.744983    1.937517
-------------+---------------------------------------------------------------
   inratday6 |        100      1.8399    .0491431         1.74239     1.93741
   inratday7 |         95    1.795158    .0426606        1.710454    1.879862
   inratday8 |         80    1.807625    .0464718        1.715125    1.900125
   inratday9 |         63    1.782381    .0521478        1.678139    1.886623
  inratday10 |         58    1.767069    .0489311        1.669086    1.865052
-------------+---------------------------------------------------------------
  inratday11 |         44    1.789773    .0536023        1.681673    1.897872
  inratday12 |         38    1.858421    .0623788         1.73203    1.984812
  inratday13 |         26    1.998077    .0730838        1.847558    2.148596
  inratday14 |         15       2.028    .1115612        1.788725    2.267275
  inratday15 |         16     1.96875    .1094717        1.735416    2.202083
-------------+---------------------------------------------------------------
  inratday16 |         14    2.087143    .0882269         1.89654    2.277745
  inratday17 |         10       2.079    .1220514          1.8029      2.3551
  inratday18 |         10       2.278    .1327638        1.977667    2.578333
  inratday19 |          8     2.02125    .1172518        1.743993    2.298506
  inratday20 |          8     2.29875    .1638863        1.911221    2.686279
-------------+---------------------------------------------------------------
  inratday21 |          7    1.888571    .0910371        1.665812    2.111331
  inratday22 |          7    2.058571    .1094669        1.790716    2.326427
  inratday23 |          4      7.1575    4.824155       -8.195115    22.51012
  inratday24 |          4        1.96     .241523        1.191366    2.728634
  inratday25 |          3        1.87    .1322875        1.300813    2.439187
-------------+---------------------------------------------------------------
  inratday26 |          1        1.85           .               .           .
  inratday27 |          3    2.043333    .2611726        .9195985    3.167068
  inratday28 |          1        1.68           .               .           .
  inratday29 |          1        1.61           .               .           .
  inratday30 |          2       2.035        .455       -3.746323    7.816323
-------------+---------------------------------------------------------------
  inratday31 |          2         2.2          .7       -6.694344    11.09434
  inratday32 |          1        1.97           .               .           .
  inratday33 |          0           .           .               .           .
  inratday34 |          1        1.75           .               .           .
  inratday35 |          0           .           .               .           .

.

Tom

[William Lisowski]

If what you want are the confidence intervals as reported by ci, not adjusted for the effects of covariates, then (a) running a regression is irrelevant, and (b) the margins command based on a regression is inappropriate, even though marginsplot can create a graph that superficially resembles the graph you want.

Below is a demonstration of technique for (a) creating a dataset containing the means and confidence intervals identical to those from the ci command using basic statistical principles and the collapse command, and (b) creating a plot from that data.

If you choose to adapt this code for your needs and have questions, you should post them in a new topic with a more appropriate title for your new questions.

sysuse auto, clear
drop if rep78==. | rep78==1
// for later comparison
ci means price if rep78==4
// calculate means and standard errors
collapse (count) n=price (mean) mean=price (semean) se=price, by(rep78)
generate cilow = mean - se*invt(n-1,0.975)
generate cihigh = mean + se*invt(n-1,0.975)
list, clean
twoway rspike cihigh cilow rep78 ||  line mean rep78 || , legend(off) ytitle("Price")

. ci means price if rep78==4

    Variable |        Obs        Mean    Std. Err.       [95% Conf. Interval]
-------------+---------------------------------------------------------------
       price |         18      6071.5    402.9585        5221.332    6921.668

. list, clean

       rep78    n      mean        se      cilow     cihigh  
  1.       2    8   5,967.6   1,265.5   2975.208   8960.042  
  2.       3   30   6,429.2     643.6   5112.924   7745.542  
  3.       4   18   6,071.5   402.959   5221.332   6921.668  
  4.       5   11     5,913   788.682   4155.707   7670.293

[Marcos Almeida]

Sorry for saying it again: please do read about the delta method, as previously recommended. This will surely solve your doubt concerning the difference between SEs as well as CIs.

Additionally, getting to the basics of each model before mixing them up will always be the best strategy.

With regards to further explanation, I bail out for several reasons: among them, the unsurpassed dispersion between the (somehow elusive) model hereby described and the commands hereby shared (for example, the clash between what seems to be a longitudinal study and the code, which conveys a cross-sectional study) , to the point that I cannot envisage a more helpful advice than the ones I’ve given so far.

[William Lisowski]

I earlier wrote, based on my understanding of what you were asking fo,r

If what you want are the confidence intervals as reported by ci, not adjusted for the effects of covariates, then (a) running a regression is irrelevant, and (b) the margins command based on a regression is inappropriate, even though marginsplot can create a graph that superficially resembles the graph you want.

On reflection, though, there's another alternative.

Perhaps you started by running margins and marginsplot because it was recommended to you to improve your presentation of your regression results, or because for some other reason you decided it was needed. And perhaps you then ran ci to compare the results to those from margins and noted the difference. And you have been asking how to get margins and marginsplot to agree with the results of ci because you do not fully understand what margins does.

In that case, then, you probably want the results of margins and marginsplot, which are an appropriate tool to show regression results, and the confidence intervals reported by ci are inappropriate, as they have nothing to do with your regression results.

I too urge you to reflect on (repeated!) advice from both Marcos and me that you need to better understand the margins command. And perhaps then you can better state your objective in the plot you want to produce. When you write

compute the confidence interval for each independent variable value

what is it that you want to produce the mean and confidence interval of - the observations of your independent variable, or some aspect of the results of your regression?

[Tom Hsiung]

I measured the drug response (anticoagulation, via a measurement called INR) every day after my patients took the anticoagulant called warfarin. So I have collect a great many of continuous quantitative data called INR. Everyday my patient would be extant venous blood, which were used to measure the INR. So I have INRs data on day 1, day 2, ..., day 35. As you may guess, I want to plot the INR and the time (day X), in the form of the mean and confidence interval of INR data. This is what I want.

I search the Internet for -margins- and I got to know it better. -Margins- command results in a mean and confidence interval of predictive response variable. Let me explain my understanding via an example. Suppose there were a linear regression with the response variable be Y and the independent be gender (category/qualitative variable) and age (continuous quantitative variable). So the linear regression formula would be written as,

Y = beta0 + beta1*gender + beta2*age

the syntax should be

regress Y i.gender age

where beta0 is the intercept and beta1, beta2 be the regression coefficients.

When I executed the command

margins i.gender

The software output margins of Y at each levels of gender, while age was set to its mean. So the margins of Y by this -margins- should be

Y(gender=0) = beta0 + beta1*0 + beta2*mean-of-age
Y(gender=1) = beta0 + beta1*1 + beta2*mean-of-age

So the computed Y values is achieved while independent variable age was "adjusted" (set to its mean). Therefore, the adjusted predicted 95% confidence intervals of Y are not the same as would be achieved by -ci mean-. But what is the significance of the adjusted predict 95% confidence intervals by -margins-? What does it predict? Is it the confidence interval of the predicted values of Y by the regression formula?

Tom

[daniel klein]

When I executed the command

margins i.gender

The software output margins of Y at each levels of gender, while age was set to its mean.

No, it does not. Read the documentation for margins more closely and also read the papers by Richard Williams that others here have referred you to. margins, by default, estimates average marginal effects not marginal effects at average values. If you want the latter, you need

margins i.gender , atmeans

Edit: Since you are running a linear model, the results will be identical. That will not be the case in general and the difference is indicated by the output that appears above the coefficient table.

But what is the significance of the adjusted predict 95% confidence intervals by -margins-? What does it predict? Is it the confidence interval of the predicted values of Y by the regression formula?

I do not fully get the question, but whatever appears in a coefficient table usually belongs together. Thus, the output shows the predicted (mean) values for Y, the associated standard error (usually derived via the delta method), the t (or z) statistic, which is calculated as Margin/SE, the corresponding p-values and confidence intervals. This is all documented with many worked examples in the manual.

Best
Daniel

[Tom Hsiung]

Thus, the output shows the predicted (mean) values for Y, the associated standard error (usually derived via the delta method), the t (or z) statistic, which is calculated as Margin/SE, the corresponding p-values and confidence intervals

So, now I did a multiple linear regression. Could I use the margin of i.gender to say gender=0 vs gender=1 are of significance?

Like, something,

P = 0.000, 2.57 vs 3.41, AD = 0.84, CI = -1.12 to -0.56

The 2.57 and 3.41 were derived from the -margins i.variable-, the 0.84 was derived from the -margins r.variable-.

Tom

[Maarten Buis]

Why do you want to use margins for that? You are using a linear model, so that difference is already directly reported in the output of regress:

. //open example data
. sysuse nlsw88, clear
(NLSW, 1988 extract)

.
. //estimate the model
. reg wage i.union ttl_exp

      Source |       SS           df       MS      Number of obs   =     1,878
-------------+----------------------------------   F(2, 1875)      =    165.97
       Model |  4905.33688         2  2452.66844   Prob > F        =    0.0000
    Residual |  27708.1055     1,875  14.7776563   R-squared       =    0.1504
-------------+----------------------------------   Adj R-squared   =    0.1495
       Total |  32613.4424     1,877  17.3753023   Root MSE        =    3.8442

------------------------------------------------------------------------------
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       union |
      union  |   1.282932   .2064182     6.22   0.000     .8780982    1.687765
     ttl_exp |   .3234277   .0192905    16.77   0.000     .2855947    .3612607
       _cons |   3.104682   .2650058    11.72   0.000     2.584944    3.624419
------------------------------------------------------------------------------

.
. // predicted wages of union and non-union members
. margins i.union, post

Predictive margins                              Number of obs     =      1,878
Model VCE    : OLS

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       union |
   nonunion  |   7.250497   .1021582    70.97   0.000     7.050141    7.450853
      union  |   8.533429   .1792379    47.61   0.000     8.181902    8.884956
------------------------------------------------------------------------------

.
. // the difference is just the same as the coefficient for 1.union in -regress-
. lincom 1.union - 0.union

 ( 1)  - 0bn.union + 1.union = 0

------------------------------------------------------------------------------
             |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         (1) |   1.282932   .2064182     6.22   0.000     .8780982    1.687765
------------------------------------------------------------------------------

[Tom Hsiung]

From the book of Introductory Statistics 10E by Weiss. Once we know the yhat, we can compute the confidence interval of conditional mean?

Concerns:
1.The yhat = the margins mean?

2. Conditional confidence interval = the confidence interval derived by -margins-?



and

[Tom Hsiung]

I made the linear regression, and I want to add the conditional means and their confidence intervals. I would like to compare two groups, so the absolute difference and its confidence interval is needed.

Tom