Three-way interaction in difference-in-differences model - interpretation and stratum-specific odds ratios

[Lotus McDougal]

I'm running a difference-in-differences model to estimate program effect using complex survey data collected at two time points in populations exposed to/not exposed to a program. I thus have a survey-adjusted logistic model predicting my outcome (anc4, dichotomized 0/1), incorporating time (time, dichotomized 0/1), treatment (tx, dichotomized 0/1[non-Program/Program]) and a number of covariates. I'm trying to identify how one particular covariate, age (age, dichotomized <18/18+), influences the program effect on my outcome, and have thus fit a three-way interaction between time, treatment and age (time x treatment being the difference in differences estimator).

Output is as follows:

. svy, subpop(finalsample): logistic anc4 i.tx##i.time##ib1.age <<additional covariates>>, base
(running logistic on estimation sample)

Survey: Logistic regression

Number of strata   =        76                  Number of obs      =     26526
Number of PSUs     =       342                  Population size    =  26579.52
                                                Subpop. no. of obs =     12930
                                                Subpop. size       = 11521.184
                                                Design df          =       266
                                                F(  27,    240)    =     19.07
                                                Prob > F           =    0.0000

-----------------------------------------------------------------------------------------------------
                                    |             Linearized
                               anc4 | Odds Ratio   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------------------------+----------------------------------------------------------------
                                 tx |
                        Non-program  |          1  (base)
                            Program  |    1.05881    .203387     0.30   0.766     .7253759    1.545513
                                    |
                            time |
                                 0  |          1  (base)
                                 1  |   1.089614   .1477585     0.63   0.527     .8342894    1.423077
                                    |
                         tx#time |
                          Program#1  |   1.873266   .4534901     2.59   0.010     1.163039    3.017204
                                    |
                        age |
                               <18  |   .7662165   .0980958    -2.08   0.038     .5954939    .9858837
                               18+  |          1  (base)
                                    |
                     tx#age |
                        Program#<18  |    1.63654   .4630892     1.74   0.083     .9374808    2.856871
                                    |
                time#age |
                             1#<18  |   1.554488   .2866231     2.39   0.017      1.08124    2.234873
                                    |
             tx#time#age |
                      Program#1#<18  |   .3786877   .1380556    -2.66   0.008     .1847337    .7762761
                                    |
  <<<Additional covariates>>>

                                    |
                              _cons |    .634593   .1625212    -1.78   0.077     .3832689     1.05072
-----------------------------------------------------------------------------------------------------

My question is two-fold. First, I want to identify the influence on program effectiveness that this particular variable (age) has. Is it thus accurate to say that the three-way interaction coefficient, 0.38, is the relative reduction in program effectiveness among participants <18 years old, or that there is 62% reduced effectiveness in this population? I know that a two-way interaction is a ratio of odds ratios, but I'm not sure how that extends to three-way interactions, and I'm not sure how to specifically interpret this three-way interaction coefficient.

Second, and I believe relatedly, how would I calculate the stratum-specific odds ratios for the difference in differences coefficient (time x treatment interaction) for participants aged <18, and those aged 18+? I would prefer not to stratify my entire model, but rather to derive those estimates from the above model that includes the three-way interaction term (so that the age stratum-specific estimates can be directly related to the change in program effectiveness discussed in my first question).

Many thanks for any advice you may have.

[Clyde Schechter]

First, I want to identify the influence on program effectiveness that this particular variable (age) has. Is it thus accurate to say that the three-way interaction coefficient, 0.38, is the relative reduction in program effectiveness among participants <18 years old, or that there is 62% reduced effectiveness in this population?

That is correct in the odds ratio metric. A three way interaction in this model is a ratio of ratios of odds ratios. Yes, it's complicated and confusing. Good luck explaining it to somebody else.

Second, and I believe relatedly, how would I calculate the stratum-specific odds ratios for the difference in differences coefficient (time x treatment interaction) for participants aged <18, and those aged 18+?

I would like to think there is a way to coax -margins- into calculating this for you, but I can't think of it. So I think we're down to relying on -nlcom-

// DID ESTIMATOR WHEN age == 0
nlcom exp(_b[1.tx#1.time]) // SHOULD MATCH RESULTS IN tx#time ROW OF REGRESSION OUTPUT

// DID ESTIMATOR WHEN age == 1
nlcom (exp(_b[1.tx#1.time] + _b[1.tx#1.time#1.age])

Note: When I say "should match results" above, I mean approximately. -nlcom- calculates standard errors differently from the way -logistic- does, so there may be a small discrepancy.

[Clyde Schechter]

Returning to what I wrote above, I notice a typo in the second -nlcom- command. There should be no ( before exp. [Or, if you prefer, you can leave that in and put an additional ) at the end.] Sorry for the error.

[Lotus McDougal]

Thank you very much, Clyde, the -nlcom- code worked perfectly!