Xtlogit, binary dep var interpretation of interaction between categorical & continuous variable

[Matthew Berg]

Hello Statalist,

Thank you for being so helpful in my research project.

I am kindly asking for your help in interpreting this xtlogit result in the odds ratio format: Adviser equals 1 if the same adviser was chosen, 0 if not (i.e., another adviser was chosen). PPD are the percentage point differences between initial and final stock prices (a continuous variable), r_sa is a continuous variable that displays the annual return figure in that investment position, and T_C signifies whether the individual was in the treatment or control group. I am especially interested in the interaction term: T_C (binary) # c.PPD

xtlogit adviser i.round i.risk c.r_sa c.PPD#i.T_C, vce(cluster CASE) or nolog

Calculating robust standard errors ...

Random-effects logistic regression                   Number of obs    =    920
Group variable: CASE                                 Number of groups =    230

Random effects u_i ~ Gaussian                        Obs per group:
                                                                  min =      4
                                                                  avg =    4.0
                                                                  max =      4

Integration method: mvaghermite                      Integration pts. =     12

                                                     Wald chi2(9)     =  82.04
Log pseudolikelihood = -444.81565                    Prob > chi2      = 0.0000

                                 (Std. err. adjusted for 230 clusters in CASE)
------------------------------------------------------------------------------
             |               Robust
      adviser| Odds ratio   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       round |
          2  |   2.267197   .6465504     2.87   0.004     1.296424    3.964894
          3  |   1.425879   .3558204     1.42   0.155     .8743188    2.325388
          4  |   2.266128   .5922252     3.13   0.002     1.357792    3.782124
             |
        risk |
          2  |   .9411477   .2300216    -0.25   0.804     .5829342    1.519484
          3  |   3.365318   .8252534     4.95   0.000     2.081095    5.442023
          4  |   4.438226   1.419868     4.66   0.000       2.3708    8.308526
             |
        r_sa |   2.93e+11   4.95e+12     1.56   0.118     .0012614    6.81e+25
             |
T_C#c.PPD |
         CG  |   .9907738   .1351929    -0.07   0.946     .7582746    1.294561
         TG  |     1.1776   .0548271     3.51   0.000     1.074898    1.290115
             |
       _cons |    1.70548   .4760463     1.91   0.056     .9868539    2.947409
-------------+----------------------------------------------------------------
    /lnsig2u |   .7635772   .2493876                      .2747865    1.252368
-------------+----------------------------------------------------------------
     sigma_u |   1.464902   .1826642                      1.147279    1.870459
         rho |   .3947783   .0595858                      .2857612    .5153752
------------------------------------------------------------------------------

I would be most grateful for any comments and advice!

Thank you very much in advance!

[Clyde Schechter]

In your model, you used the # operator for the interaction, but did not include the "main" effects separately. Under these circumstances, and given that you want to interpret in the odds ratio metric, you can just read it off the regression output. In the CG group, the odds of advisor shrinks by a factor of 0.991 (to 3 decimal places) per unit difference in PPD, and in the TG group grows by a factor of 1.178. Do note that the confidence intervals for these results overlap extensively, so we can't say much about the difference between these two odds ratios based only on this information. For that you would have to use -lincom- (don't forget to add the -or- option) to specifically see the confidence interval around the difference between them.

[Matthew Berg]

Thank you very much, Clyde! I appreciate your detailed answer! May I kindly ask for your opinion on the suitability of the xtlogit model? Specifically, what concerns me is the assumption of a random effects model that time-invariant independent variables should not influence the dependent variable. But I believe that the treatment group (considered time invariant because set once at the beginning of the experiment and is not changed) will affect my binary dependent variable.

As a remedy, one should use a standard logit model (without the xt commands) with vce(cluster ID) without any random effects or fixed effects options. I've heard about this once or twice briefly but haven't found any credible resources confirming this, so I would be most grateful for your input.

I would like to hold on to my -xt models as they account for omitted variable bias, as you've kindly pointed out in this thread: https://www.statalist.org/forums/for...-and-r-squared

Thank you very much in advance for your time and effort!

[Clyde Schechter]

The worry with random effects models is that the error terms may fail to be independent of the predictors due to the confounding effect of variables not included in the model. Your treatment variable is included in the model. So that one is not an issue. The concern is that there may be other influences not accounted for in the model that confound the treatment:outcome relationship, and thereby bias the effect estimate. Those other influences, by the way, could be time-varying or time-invariant. The reason for the focus on the time-invariant ones is that a fixed-effects model can eliminate the problem they cause, whereas it does not eliminate the influence of time-varying ones.

Another important point is that if your treatment vs control assignments were randomized, then you have no issue, because the randomization makes the treatment variable independent of other things (with high probability in a sample the size of yours).

So, to the extent that you want to improve your model, if treatment was not randomized, it would be by adding more covariates that influence the outcome and might be imbalanced across the treatment vs control divide.

When you have panel data, using cross-sectional analyses like -reg- with clustered standard errors is a second or third-rate way to go. I would reserve that approach for situations where for technical reasons you cannot use a true panel-data estimator.

[Matthew Berg]

Thank you very much for your reassurance, Clyde! Yes, indeed, the assignment to the treatment and control group is random.

Please allow me once more to come back to your statement from https://www.statalist.org/forums/for...-and-r-squared (#2)

[QUOTE=Clyde Schechter;n1421512]

3. There are two circumstances where one would revert to -logit, vce(cluster panel)-, however. One is when the output of -xtlogit, fe- shows that in fact there is only negligible variance at the panel level. In that case any time-invariant attributes of the panels will not be confounding variables, so there is no omitted variable bias (from these attributes) to adjust for, and the -logit- model is simpler.

3 (cont'd.) The other circumstance is more complicated. It is quite common for models that include a covariate or set of covariates to produce very different results from a model that excludes them. (An -xtlogit, fe- can be thought of as including covariates representing the panels--that's not literally true, but for present purposes it works.) The differences can be dramatic, including opposite signs. This is known as Simpson's paradox and it is a direct reflection of omitted variable "bias." However, there's a catch. Depending on the actual causal relations, sometimes it is the model with the variables omitted that represents the true causal effect and the inclusion of the variables results in bias. This occurs, for example, if the covariates in question lie on the causal path between the predictor of interest and the outcome. In this situation, it is the -logit- model that is correct, and -xtlogit- would be wrong.
/QUOTE]

Regarding the first 3. Could you kindly please share some evidence of negligible variance at the panel level?

Regarding the second 3: Can't I argue that I have no covariates within an experiment (ie, a controlled environment)?

Would it be sensible to run both regressions once with xtset and xtlogit, vce(cluster ID) and the other just with logit, vce (cluster ID) and compare the coefficients and determine whether the results/coefficients are similar?

Thank you so much for your insights and time!

[Clyde Schechter]

Regarding the first 3. Could you kindly please share some evidence of negligible variance at the panel level?

The simplest way to do this is to look at the output for rho at the bottom of the regression table. Negligible variance at the panel level means rho very close to zero. In the results you show, rho is 0.39, which is by no means close to zero.

Regarding the second 3: Can't I argue that I have no covariates within an experiment (ie, a controlled environment)?

Yes. With an experimental design, you don't have to worry about the limitations of the random effects model that arise from confounding by unmeasured variables. Your sample size is large enough that the law of large numbers gives you a high probability that confounding will not exist. But this implies that you should stick with your -xtlogit- model. If there is no confounding, then there is no question about the confounded model being a better model of the causal relationships than the adjusted model.

I would not run a plain -logit- model on this data.

[Matthew Berg]

Thank you so much! I appreciate your help greatly!

[Jeff Wooldridge]

I'll differ a bit with Clyde here. The random effects specification imposes a very special serial correlation pattern, and having an experiment does not rule out complicated serial dependence. So I actually would try pooled logit and cluster the standard errors. I doubt that, when you compare, say, marginal effects, the answers will be different. But the pooled logit is, technically, more robust than xtlogit, re because logit is consistent regardless of the serial correlation pattern in the underlying shocks.

[Matthew Berg]

Dear Jeff,

Thank you very much for your comment.

May I please ask for some clarification:

Do you mean the serial correlation of one participant in the four rounds (i.e., round 1 is serially correlated with round 2, which is correlated with round 3, which is correlated with round 4 for the same participant)? Or do you mean serial correlation across panel units? Per
https://www.stata.com/meeting/uk17/s...17_Wursten.pdf, useful commands are pwcorrf and xtcdf.

I would like to add that the four rounds are inherently different by risk category: The idea is that every round has a distinct risk profile attached to it (very low to very high). The participant is informed at the beginning of the experiment that the four rounds are independent of each other. The individual should decide whether they keep the same investment adviser or not.
For example, in the very low risk group, the return was 2.5% pa, and the percentage points difference between the ending and beginning stock price was +5: The participant should decide whether s/he would keep the investment adviser.

In the second round, let's assume the participant is in the high risk category and the return was actually -15%. Again the same question: Does the participant want to keep this investment adviser?

The allocation of the risk categories to the rounds is random, ie, round 1 is not always risk category 1, etc.

I would argue that the four rounds do not have serial dependence, as they are completely different; just the task for the participant is the same.

Since I have a binary dependent variable (and therefore logistic regression), I believe I cannot use xtserial (as this is only for linear panel data) to check for serial correlation within panel data. Nonetheless, I ran

 xtserial adviser round risk r_sa PPD T_C, output

and obtained the following:

xtserial adviser round risk r_sa PPD T_C, output

Linear regression                               Number of obs     =        690
                                                F(4, 229)         =      17.82
                                                Prob > F          =     0.0000
                                                R-squared         =     0.0989
                                                Root MSE          =     .52647

                                 (Std. err. adjusted for 230 clusters in CASE)
------------------------------------------------------------------------------
             |               Robust
    D.adviser | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
       round |
         D1. |    .032908   .0116376     2.83   0.005     .0099775    .0558384
             |
        risk |
         D1. |   .0690625   .0133844     5.16   0.000     .0426902    .0954348
             |
        r_sa |
         D1. |   2.084726   2.520437     0.83   0.409    -2.881485    7.050938
             |
      PPD |
         D1. |   .0240354   .0073651     3.26   0.001     .0095235    .0385474
             |
         T_C |
         D1. |          0  (omitted)
------------------------------------------------------------------------------

Wooldridge test for autocorrelation in panel data
H0: no first order autocorrelation
    F(  1,     229) =      0.073
           Prob > F =      0.7869

In my opinion, these results mean that we fail to accept the Null Hypothesis and conclude serial correlation is not a problem.

If I understand correctly, you say I should not use xtlogit (with default re) models to analyze my panel data as serial correlation is present. Therefore, I should use logit (without the xt setting) but pool standard errors with vce(cluster ID). Is this correct?

I'm very sorry, I thought the -xt commands were specifically designed for panel data, and just using logit with vce(cluster ID) will disregard the panel data structure.

I am most grateful for your comments and insights.

Thank you very much in advance!