Question: Creating and Modeling Within- and Between-Level Effects using Longitudinal Data

[John Brent]

Greetings, All:

Hope all is well. I am looking for some guidance/syntax that will model the extent to which within (time variant) and between (time-invariant) variance associated with an independent variable (bmtx below) impact the outcome. The syntax currently be used is

bys ID_NEW: egen bmtxbtw = mean(bmtx)
gen bmtxwin = bmtx - bmtxbtw

After reshaping 3 waves of data (long) and using the xtsum command, I understand that I should find “0” for the within-person standard deviation for the between-person effect, and “0” for the between-person standard deviation for the within-person effect. However, the output suggest I may be doing something wrong – or, perhaps there is little/no within variance (see below for output).

Thank you in advance for you time and assistance.

[Clyde Schechter]

I understand that I should find “0” for the within-person standard deviation for the between-person effect, and “0” for the between-person standard deviation for the within-person effect.

That understanding is correct.

However, the output suggest I may be doing something wrong – or, perhaps there is little/no within variance (see below for output).

No, the output suggests that there are very tiny rounding errors in the calculation of bmtx_win. Remember that Stata's calculations are finite precision and get truncated or rounded to just float precision. Your worst departure from 0 in bmtx_win's between variation is 2 x 10^-8 in a variable whose overall mean is 0.07. And the order of magnitude of bmtx_win is 10^-9. I don't think I would worry about that. These rounding errors are probably far less than the measurement error in your data to start with. If you are really bothered by them at this magnitude, you could re-do the calculations specifying double precision for bmtx_btw and bmtx_win. That will probably bring the rounding errors down to an order of magnitude of something like 10^-15. If that doesn't put your mind at ease, I don't know what to tell you.

Added: By the way, if you are running Stata Now (version 18.5) you don't need to calculate these variables. You can use the -cre- option of -xtreg- in your analysis instead and Stata will do the grunt work for you. With earlier versions of Stata, you can install Francisco Perales & Reinhard Schunck's -xthybrid- command, available from Stata Journal (SJ18-4: st0468_1), and that will also serve the purpose at hand.

[Erik Ruzek]

Agreed with Clyde. One thing to keep in mind is that the xtreg, cre and xthybrid commands do not allow for random slope models, in which you allow the slope for a given within-person predictor to vary across persons (ID_NEW). For that you would need to use mixed.

[John Brent]

Thank you, both, for you response, insight, and guidance - it is much appreciated. I can 100% live with and understand the rounding errors. But, the output make me question whether modeling the between and within effects separately (to parse out where the influence is from the overall variable, as a reviewer questioned) is worth it? Thoughts? Thank you once more.

[Clyde Schechter]

I don't see anything in the output shown in #1 that bears on the usefulness of modeling within and between effects separately. I think the only way you can know that is to actually do it and see the results. If the between effects and within effects turn out to be equal (or close enough for practical purposes) then, yes, it would make sense to then step back and go to a simpler model that doesn't distinguish between them. But I don't think the descriptive data shown in -xtsum- output enables you to foresee whether that will happen or not.

And, to be clear, within and between effects are often very different, possibly even in opposite directions, and an analysis of longitudinal data that just smashes them together without determining first that they don't appreciably differ is just producing meaningless, misleading results.

[John Brent]

Thank you, once more - truly appreciate your guidance. Hope you have a great day and year to come.

[Erik Ruzek]

The question of whether the independent variable is having a differential effect at one level or another can be empirically examined. You can test whether the group-level effect is the same or different from the within-group effect using a hybrid or Mundlak model in which you include the group mean of the predictor in the mixed model in addition to the uncentered predictor. The p-value associated with the coefficient for the group mean predictor is an exact test of whether the within and between coefficients are equivalent (null hypothesis: they are not different). If you have multiple such predictors in your model, you can run a post-estimation test of whether the included group mean predictors are jointly equal to 0 using the testparmcommand. There are a number of good writeups on this topic. Bell & Jones 2015 is a good one.