ANOVA error r(422); "could not determine between-subject basic unit; use bseunit() option" Using bseunit() option, but get this error anyway

Matt McMahon

Join Date: Aug 2017

Posts: 11
#1

ANOVA error r(422); "could not determine between-subject basic unit; use bseunit() option" Using bseunit() option, but get this error anyway

29 May 2019, 10:51

Hello,

I'm new to using the ANOVA command (it is not common in my field). A reviewer for a journal submission asked me to use ANOVA as a robustness test to my main specification. It likely will not go in the final version of the paper, but I need to run it correctly in order to appease the reviewer. I think I've found a solution (and potentially a cause) to my issue, but I want to report the error here anyway because it seems like an issue with the ANOVA command itself. Even with my solution, it seems that this is the wrong error to be popping up, so I think the ANOVA command needs to be tweaked so that it provides the correct error. I'm using the current version of Stata SE, 15.1.

The basic problem is that I'm attempting to run ANOVA with panel data from an experiment. (I'll add specifics in the next paragraph.) When I first specify my repeated variable(s) using the repeated() option, the command is unable to automatically detect the bse ("between-subject error") term. I'm met with an error: r(421); could not determine between-subject error term; use bse() option. I then added the bse() option and tried it again. This time I was met with a different error: r(422); could not determine between-subject basic unit; use bseunit() option. So I then added the bseunit() option and tried it again. However, I was then met with the exact same error. It's telling me that it needs me to specify an option that I'm already specifying!

To give some background, my data is from an experiment. (I'll just use a subset that reproduces the error as a minimum working example, so these numbers don't actually represent my full experimental data.) I have 38 total subjects who played a game either 3 or 9 total times (unbalanced panel), for a total of 192 observations. Note that the dependent variable is actually a ratio of two other variables. For 20 total observations, the bottom variable used in the ratio is 0, and thus those 20 observations have a "missing" value for the dependent variable. Those 20 are (automatically) dropped from all regressions below, leaving n=172.

Subjects are nested within 6 total sessions (with subjects playing 3 repetitions each during the first 4 sessions, and subjects within the last 2 sessions playing 9 repetitions each). Subjects are indexed by the variable SubjectID, sessions are indexed by the variable SessionID, and repetitions of the game are indexed by the variable Period. Thus, I use xtset SubjectID Period at the beginning of my code (both for my standard analysis and for the ANOVA analysis).

I should note that because I limited the data to a minimum working example (eliminating all but 2 treatments), the session numbering skips around (SessionID equals 4, 5, 6, 8, 10, or 11). For the same reason, repetitions do not line up exactly across sessions. For example, for SessionID==4, Period may equal 10, 11, or 12, while for SessionID==10, Period may equal any integer from 1 to 9 (inclusive). The full data is located at the bottom of the post.

There are two treatments, "N" (baseline) and "D". Subjects who played 3 total repetitions played only 1 of the treatments, meaning that treatment is nested within subject (in turn nested within session) for them. Subjects who played 9 total repetitions played both treatments (with the order in which they played them flipped across the two relevant sessions), meaning that treatment is not nested within subject (nor session) for them. So overall, treatment is not nested within subject, nor even session. (I specify all of this because properly specifying nesting seems to matter with regards to getting these errors.)

The main regression specification in my paper uses the following code (again, edited down to provide a minimum working example, hence the "if" statements):

Code:

xtreg LiqPerc1_2_B TreatD if Agent==1 & TreatAV==1, re cluster(SessionID)

This runs a panel regression on the dependent variable LiqPerc1_2_B using a constant and a treatment dummy TreatD that equals 1 for treatment D and 0 for treatment N. The coefficient estimate for TreatD tells the marginal impact of switching from the baseline to treatment D, which is the estimate of interest. The re option specifies that the model include random effects at the subject level (SubjectID), and the cluster() option specifies that standard errors be clustered at the session level (SessionID).

The reviewer wants me to run an ANOVA model as an alternative specification to test for within-session issues. From reading through the ANOVA help text, the examples provided in the r.pdf file, and various online forums, I had thought that my corresponding ANOVA specification should be the following:

Code:

anova LiqPerc1_2_B TreatD / Period SessionID if Agent==1 & TreatAV==1, repeated(SessionID)

However, as previously mentioned, this produces the error r(421); could not determine between-subject error term; use bse() option. I then specified that option, using the following:

Code:

anova LiqPerc1_2_B TreatD / Period SessionID if Agent==1 & TreatAV==1, repeated(SessionID) bse(SessionID)

This time, I get a new error: r(422); could not determine between-subject basic unit; use bseunit() option. So I added that option, using the following:

Code:

anova LiqPerc1_2_B TreatD / Period SessionID if Agent==1 & TreatAV==1, repeated(SessionID) bse(SessionID) bseunit(SessionID)

This results in the same exact error. I'm getting an error code telling me that I need to specify the bseunit() option, even though I'm already specifying it. While I now believe that this is the incorrect specification, hence the fact that the model won't run, I believe that the specific error code needs to be changed to one that more accurately reflects the problem/provides instructions for a solution.

To that point, I essentially used trial and error to figure out what is hopefully the proper model specification. My final specification choice was the following:

Code:

anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(Period) grouping(SessionID)

Though, I should note that the grouping() command is not necessary to make it run; this will also run without errors:

Code:

anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(Period)

Ultimately, I guess that I have the following questions:
1. What is causing this error to appear, and can it be fixed?
2. Is my final specification correctly adapting my paper's main specification to the ANOVA model?

Thank you for your help.

Best,
Matt

PS, in case it helps with diagnosis, here is a list of some other codes I tried that resulted in the same (or similar) errors:
-anova LiqPerc1_2_B TreatD / SessionID if Agent==1 & TreatAV==1, repeated(SessionID) (r(422))
--anova LiqPerc1_2_B TreatD / SessionID if Agent==1 & TreatAV==1, repeated(SessionID) bseunit(SessionID) (r(422))
-anova LiqPerc1_2_B TreatD / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) (r(422))
--anova LiqPerc1_2_B TreatD / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bseunit(SessionID) (r(422))
--anova LiqPerc1_2_B TreatD / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bseunit(SubjectID) (r(422))
-anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) (r(421))
--anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SubjectID) (r(198); bse() term not found in model, though it seems to me that it is)
--anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SessionID) (r(198); bse() term not found in model, though it seems to me that it is)
--anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bseunit(SubjectID) (r(421))
---anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SessionID) bseunit(SubjectID) (r(198); bse() term not found in model, though it seems to me that it is)
---anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SubjectID) bseunit(SubjectID) (r(198); bse() term not found in model, though it seems to me that it is)
--anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bseunit(SessionID) (r(421))
---anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SessionID) bseunit(SessionID) (r(198); bse() term not found in model, though it seems to me that it is)
---anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SubjectID) bseunit(SessionID) (r(198); bse() term not found in model, though it seems to me that it is)
-anova LiqPerc1_2_B TreatD / Period SessionID / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) (r(421))
--anova LiqPerc1_2_B TreatD / Period SessionID / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SessionID) (r(422))
---anova LiqPerc1_2_B TreatD / Period SessionID / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SessionID) bseunit(SessionID) (r(422))
---anova LiqPerc1_2_B TreatD / Period SessionID / SubjectID|SessionID if Agent==1 & TreatAV==1, repeated(SubjectID SessionID) bse(SessionID) bseunit(SubjectID) (r(422))
-anova LiqPerc1_2_B TreatD / Period SessionID if Agent==1 & TreatAV==1, repeated(SessionID) (r(421))
--anova LiqPerc1_2_B TreatD / Period SessionID if Agent==1 & TreatAV==1, repeated(SessionID) bse(Period) (r(422))
---anova LiqPerc1_2_B TreatD / Period SessionID if Agent==1 & TreatAV==1, repeated(SessionID) bse(Period) bseunit(Period) (r(422))

PPS, here are the relevant variables for all 120 observations used in the minimum working example (that is, if Agent==1 & TreatAV==1). I saw on the "help" tab that posting .dta files is not recommended, so I figured I was supposed to post it long-form. I hope that it's not too long.

list SessionID SubjectID Period TreatD LiqPerc1_2_B in 1/120, separator(3)

Sessio~D Subjec~D Period TreatD LiqPer~B
-
1. 4 61 10 0 15
2. 4 61 11 0 4
3. 4 61 12 0 .
4. 4 64 10 0 .
5. 4 64 11 0 37.5
6. 4 64 12 0 .
7. 4 66 10 0 50
8. 4 66 11 0 100
9. 4 66 12 0 50
10. 4 68 10 0 50
11. 4 68 11 0 50
12. 4 68 12 0 45
13. 4 71 10 0 100
14. 4 71 11 0 .
15. 4 71 12 0 37.5
16. 4 72 10 0 50
17. 4 72 11 0 .
18. 4 72 12 0 52
19. 5 82 10 1 100
20. 5 82 11 1 100
21. 5 82 12 1 100
22. 5 83 10 1 30
23. 5 83 11 1 29.23077
24. 5 83 12 1 50
25. 5 85 10 1 87.5
26. 5 85 11 1 75
27. 5 85 12 1 100
28. 5 88 10 1 36.36364
29. 5 88 11 1 42.85714
30. 5 88 12 1 50
31. 5 90 10 1 33.33333
32. 5 90 11 1 .
33. 5 90 12 1 0
34. 5 91 10 1 20
35. 5 91 11 1 16.66667
36. 5 91 12 1 14.28571
37. 5 92 10 1 100
38. 5 92 11 1 50
39. 5 92 12 1 .
40. 5 95 10 1 22.22222
41. 5 95 11 1 18.18182
42. 5 95 12 1 28.57143
43. 6 101 10 1 0
44. 6 101 11 1 100
45. 6 101 12 1 47.61905
46. 6 102 10 1 12.5
47. 6 102 11 1 82
48. 6 102 12 1 .
49. 6 104 10 1 70
50. 6 104 11 1 85.71429
51. 6 104 12 1 83.33334
52. 6 107 10 1 25
53. 6 107 11 1 0
54. 6 107 12 1 30
55. 6 109 10 1 91.66666
56. 6 109 11 1 .
57. 6 109 12 1 92.85714
58. 6 111 10 1 .
59. 6 111 11 1 12.5
60. 6 111 12 1 25
61. 8 141 4 0 40
62. 8 141 5 0 52.94118
63. 8 141 6 0 42.85714
64. 8 143 4 0 25
65. 8 143 5 0 33.33333
66. 8 143 6 0 37.5
67. 8 145 4 0 71.42857
68. 8 145 5 0 100
69. 8 145 6 0 55.55556
70. 8 148 4 0 97.5
71. 8 148 5 0 8.333333
72. 8 148 6 0 6.666667
73. 8 149 4 0 50
74. 8 149 5 0 25
75. 8 149 6 0 40
76. 10 181 1 0 33.33333
77. 10 181 2 0 34.28571
78. 10 181 3 0 20
79. 10 181 4 1 35
80. 10 181 5 1 30
81. 10 181 6 1 40
82. 10 181 7 0 25
83. 10 181 8 0 15.09434
84. 10 181 9 0 25
85. 10 182 1 0 66.66666
86. 10 182 2 0 50
87. 10 182 3 0 50
88. 10 182 4 1 50
89. 10 182 5 1 57.14286
90. 10 182 6 1 50
91. 10 182 7 0 50
92. 10 182 8 0 60
93. 10 182 9 0 50
94. 10 184 1 0 30
95. 10 184 2 0 20
96. 10 184 3 0 20
97. 10 184 4 1 .
98. 10 184 5 1 40
99. 10 184 6 1 41.66667
100. 10 184 7 0 33.33333
101. 10 184 8 0 21.42857
102. 10 184 9 0 .
103. 10 187 1 0 50
104. 10 187 2 0 75
105. 10 187 3 0 66.66666
106. 10 187 4 1 50
107. 10 187 5 1 80
108. 10 187 6 1 66.66666
109. 10 187 7 0 60
110. 10 187 8 0 75
111. 10 187 9 0 83.33334
112. 10 188 1 0 66.66666
113. 10 188 2 0 50
114. 10 188 3 0 60
115. 10 188 4 1 55
116. 10 188 5 1 40
117. 10 188 6 1 .
118. 10 188 7 0 41.66667
119. 10 188 8 0 81.25
120. 10 188 9 0 0

Last edited by Matt McMahon; 29 May 2019, 10:52. Reason: Edited to add tags
Tags: anova, bse, bseunit, r422, repeated

William Lisowski

Join Date: Dec 2014
Posts: 10150

29 May 2019, 14:08

Welcome to Statalist.

In case someone knowledgeable with the anova command wants to replicate your work. I've taken the effort to present the 120 observations you show using the dataex command, as recommended in the Statalist FAQ linked to from the top of the page, and well-documented in the output of help dataex. In your case, the appropriate dataex command would have been

Code:

dataex SessionID SubjectID Period TreatD LiqPerc1_2_B i if Agent==1 & TreatAV==1, count(120)

Code:

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte SessionID int SubjectID byte(Period TreatD) float LiqPerc1_2_B
 4  61 10 0       15
 4  61 11 0        4
 4  61 12 0        .
 4  64 10 0        .
 4  64 11 0     37.5
 4  64 12 0        .
 4  66 10 0       50
 4  66 11 0      100
 4  66 12 0       50
 4  68 10 0       50
 4  68 11 0       50
 4  68 12 0       45
 4  71 10 0      100
 4  71 11 0        .
 4  71 12 0     37.5
 4  72 10 0       50
 4  72 11 0        .
 4  72 12 0       52
 5  82 10 1      100
 5  82 11 1      100
 5  82 12 1      100
 5  83 10 1       30
 5  83 11 1 29.23077
 5  83 12 1       50
 5  85 10 1     87.5
 5  85 11 1       75
 5  85 12 1      100
 5  88 10 1 36.36364
 5  88 11 1 42.85714
 5  88 12 1       50
 5  90 10 1 33.33333
 5  90 11 1        .
 5  90 12 1        0
 5  91 10 1       20
 5  91 11 1 16.66667
 5  91 12 1 14.28571
 5  92 10 1      100
 5  92 11 1       50
 5  92 12 1        .
 5  95 10 1 22.22222
 5  95 11 1 18.18182
 5  95 12 1 28.57143
 6 101 10 1        0
 6 101 11 1      100
 6 101 12 1 47.61905
 6 102 10 1     12.5
 6 102 11 1       82
 6 102 12 1        .
 6 104 10 1       70
 6 104 11 1 85.71429
 6 104 12 1 83.33334
 6 107 10 1       25
 6 107 11 1        0
 6 107 12 1       30
 6 109 10 1 91.66666
 6 109 11 1        .
 6 109 12 1 92.85714
 6 111 10 1        .
 6 111 11 1     12.5
 6 111 12 1       25
 8 141  4 0       40
 8 141  5 0 52.94118
 8 141  6 0 42.85714
 8 143  4 0       25
 8 143  5 0 33.33333
 8 143  6 0     37.5
 8 145  4 0 71.42857
 8 145  5 0      100
 8 145  6 0 55.55556
 8 148  4 0     97.5
 8 148  5 0 8.333333
 8 148  6 0 6.666667
 8 149  4 0       50
 8 149  5 0       25
 8 149  6 0       40
10 181  1 0 33.33333
10 181  2 0 34.28571
10 181  3 0       20
10 181  4 1       35
10 181  5 1       30
10 181  6 1       40
10 181  7 0       25
10 181  8 0 15.09434
10 181  9 0       25
10 182  1 0 66.66666
10 182  2 0       50
10 182  3 0       50
10 182  4 1       50
10 182  5 1 57.14286
10 182  6 1       50
10 182  7 0       50
10 182  8 0       60
10 182  9 0       50
10 184  1 0       30
10 184  2 0       20
10 184  3 0       20
10 184  4 1        .
10 184  5 1       40
10 184  6 1 41.66667
10 184  7 0 33.33333
10 184  8 0 21.42857
10 184  9 0        .
10 187  1 0       50
10 187  2 0       75
10 187  3 0 66.66666
10 187  4 1       50
10 187  5 1       80
10 187  6 1 66.66666
10 187  7 0       60
10 187  8 0       75
10 187  9 0 83.33334
10 188  1 0 66.66666
10 188  2 0       50
10 188  3 0       60
10 188  4 1       55
10 188  5 1       40
10 188  6 1        .
10 188  7 0 41.66667
10 188  8 0    81.25
10 188  9 0        0
end

Last edited by William Lisowski; 29 May 2019, 14:14.

Comment

Matt McMahon

Join Date: Aug 2017

Posts: 11
#3

05 Jun 2019, 10:42

Originally posted by William Lisowski View Post

Welcome to Statalist.

In case someone knowledgeable with the anova command wants to replicate your work. I've taken the effort to present the 120 observations you show using the dataex command, as recommended in the Statalist FAQ linked to from the top of the page, and well-documented in the output of help dataex.

Thank you! I browse these forums a lot, but I'm still new to posting. I really appreciate you taking the time to not only point out my mistake, but also to actually show me (and post) the corrected version. (I also just discovered that my email notifications must be turned off, hence my late response. Sorry about that!)
Comment
William Lisowski

Join Date: Dec 2014

Posts: 10150
#4

05 Jun 2019, 19:28

I regret that nobody has offered any interpretation of what you're seeing. Below is ready-to-run code that when applied to the sample data in post #2 reproduces your experience - the first three anovas fail while the final two do not.

Code:

xtset SubjectID Period xtreg LiqPerc1_2_B TreatD, re cluster(SessionID) capture noisily anova LiqPerc1_2_B TreatD / Period SessionID, /// repeated(SessionID) capture noisily anova LiqPerc1_2_B TreatD / Period SessionID, /// repeated(SessionID) bse(SessionID) capture noisily anova LiqPerc1_2_B TreatD / Period SessionID, /// repeated(SessionID) bse(SessionID) bseunit(SessionID) capture noisily anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID, /// repeated(SubjectID SessionID) bse(Period) grouping(SessionID) capture noisily anova LiqPerc1_2_B TreatD / Period SubjectID|SessionID, /// repeated(SubjectID SessionID) bse(Period)

When we look at the final two anova commands that work, we see as you did that the failure to include SubjectID in the model as a repeated measure nested within SessionID led to the problem with the bseunit() option.

Because indeed the error message after the third anova is insensitive to the fact that you are already doing what it instructs you to do, you could consider sharing this with Stata Technical Services and asking them if they can explain further. That would address your first question.

Your second question might get a response if you were to repost the data from post #2 and the code from above to a new topic with a more focused title such as "comparing xtreg and repeated mesasures anova", and with a briefer explanation. This might pique the interest of someone who is more interested in statistics than in syntax. If you do that, be sure to explain that what you are presenting was extracted from this topic (provide a link) after half of your problems were addressed. Statalist generally discourages reposting, bumping, and the like, but I think in this case you had two questions only one of which was in the title. Readers do make judgements about what they might be interested in, and for that matter, if we get a good explanation there, it will be easier for a later reader with a similar problem to find.
1 like
Comment
Matt McMahon

Join Date: Aug 2017

Posts: 11
#5

06 Jun 2019, 08:21

Originally posted by William Lisowski View Post

[...]

Because indeed the error message after the third anova is insensitive to the fact that you are already doing what it instructs you to do, you could consider sharing this with Stata Technical Services and asking them if they can explain further. That would address your first question.

Thank you for your help! That looks like a pretty good explanation of what must be going on under the hood in general. I will at least mention it to them, even though I personally don't need the resolution anymore at this point.

Originally posted by William Lisowski View Post

Your second question might get a response if you were to repost the data from post #2 and the code from above to a new topic with a more focused title such as "comparing xtreg and repeated mesasures anova", and with a briefer explanation. This might pique the interest of someone who is more interested in statistics than in syntax. If you do that, be sure to explain that what you are presenting was extracted from this topic (provide a link) after half of your problems were addressed. Statalist generally discourages reposting, bumping, and the like, but I think in this case you had two questions only one of which was in the title. Readers do make judgements about what they might be interested in, and for that matter, if we get a good explanation there, it will be easier for a later reader with a similar problem to find.

I will also try this. Thank you for your advice!
Comment
Dave Airey

Join Date: Apr 2014

Posts: 390
#6

07 Jun 2019, 13:11

In the example data, and even in your description of the experiment, for some of your subjects, treatment is a between subjects factor, where a subject sees only one or the other treatment level. For other subjects treatment is a within subject factor, where each subject sees all treatment levels. This is going to cause anova a problem, as TreatD is a repeated variable for some of the data only. I don't think the problem is with anova, but with your design.
2 likes
Comment
Joseph Coveney

Join Date: Apr 2014

Posts: 4129
#7

08 Jun 2019, 03:33

First, about -anova-'s balking, here:

Code:

anova LiqPerc1_2_B SessionID / SubjectID|SessionID /// Period TreatD Period#TreatD /// SessionID#Period SessionID#TreatD SessionID#Period#TreatD, /// repeated(TreatD Period)

Second,

Originally posted by Matt McMahon View Post

Note that the dependent variable is actually a ratio of two other variables. For 20 total observations, the bottom variable used in the ratio is 0, and thus those 20 observations have a "missing" value for the dependent variable. Those 20 are (automatically) dropped from all regressions below, leaving n=172.

?

It sounds as if your numerator is the actual outcome variable and the denominator is some kind of adjustment. Have you considered just using the numerator as the outcome variable and putting the denominator in the model as an -offset()-? -xtreg , re- doesn't offer an -offset()- option, but -meglm- does.

Third,

I have 38 total subjects who played a game either 3 or 9 total times (unbalanced panel)

With only certain exceptions, a mixed-model ANOVA requires balanced data for valid test statistics.

Fourth,

I use xtset SubjectID Period at the beginning of my code (both for my standard analysis and for the ANOVA analysis). . . . The main regression specification in my paper uses the following code (again, edited down to provide a minimum working example, hence the "if" statements):

xtreg LiqPerc1_2_B TreatD if Agent==1 & TreatAV==1, re cluster(SessionID)

You do understand that that model does not include Period at all, right? Is that your intention? Because if it is, then the ANOVA model corresponding to it is much simplified:

Code:

anova LiqPerc1_2_B TreatD SubjectID

Last,

The reviewer wants me to run an ANOVA model as an alternative specification to test for within-session issues.

It's not clear what issues you're trying to test for, but Session isn't part of your -xtreg , re- model.
2 likes
Comment
Dave Airey

Join Date: Apr 2014

Posts: 390
#8

08 Jun 2019, 11:20

Hi Joseph,

You are right--my comment was made looking at the design, which seems odd, without running code. Stata doesn't complain when I run the models with just the two repeated variables, despite the fact that repeated levels are not all measured for most subjects. I note the two within subject factors' levels are not crossed (everyone has an ABA exposure to treatment over periods in SessionID == 10). In the code output of your model and the models below, there is a problem with terms having 0 df which I'm assuming is because we have a subset of the data posted?

Anyway it would be of interest if Matt posted back on application of your syntax to his full dataset and your questions.

Code:

tabulate SessionID TreatD anova LiqPerc1_2_B SubjectID Period / SubjectID#Period TreatD / SubjectID#TreatD /// Period#TreatD, repeated(Period TreatD) dropemptycells anova LiqPerc1_2_B SubjectID Period / SubjectID#Period TreatD / SubjectID#TreatD /// Period#TreatD if SessionID == 10, repeated(Period TreatD) dropemptycells anova LiqPerc1_2_B SubjectID Period / SubjectID#Period TreatD / SubjectID#TreatD /// Period#TreatD if SessionID != 10, repeated(Period TreatD) dropemptycells
Comment
Joseph Coveney

Join Date: Apr 2014

Posts: 4129
#9

08 Jun 2019, 17:14

Originally posted by Dave Airey View Post

there is a problem with terms having 0 df which I'm assuming is because we have a subset of the data posted?

That's my guess, too.
Comment

Matt McMahon

Join Date: Aug 2017
Posts: 11

#10

20 Jun 2019, 13:00

[Post 1 of 2]

Hey Dave and Joseph,

I apologize for my delayed response. I was on vacation for a week and a half, but I’m back in the office now.

This post went just a bit past the character limit, so I've had to split it into two.

To the point about there being a problem because I’m using a subsample, I’ll use my “full” sample/model here and show the output. The experiment is essentially a 2x2x2x2 design, though we only ran 12 of the 16 possible treatment cells. However, for this question, we are eliminating one dimension (I’ll use -if TreatV==1- in all my code) and are ignoring one dimension (aka, we are pooling across the “low” and “high” outcomes that are determined during later stages of the game, thus not impacting the variables of interest here). So we have dimension 1, which we call treatment A or F, and dimension 2, which we call N or D. We ultimately care about the marginal impact of switching from N to D within the A and F regimes, which is why our main specification is arranged as it is. (The coefficient estimates for -TreatAD- and -TreatFD- are the estimates of interest.) (Side note: I really wish that -xtreg- with -re- would allow the -nocons- option for an even cleaner specification, but that’s a different issue.) I should also note that each observation actually involves a pair of subjects, so I’ve restricted everything (including the -tabulate- command) to just the relevant half of the pair (-if Agent==1-).

With those caveats, I’ll see if I can go through all the points you both made in sequence.

First, for comparison, here are the results from my main specification.

Code:

. /*** Using the Full Sample ***/
. /** Main Specification (from the Paper) **/
. xtreg LiqPerc1_2_B TreatAD TreatF TreatFD if TreatV==1 & Agent==1, re cluster(SessionID)
 
Random-effects GLS regression                   Number of obs     =        528
Group variable: SubjectID                       Number of groups  =         66
 
R-sq:                                           Obs per group:
     within  = 0.0310                                         min =          1
     between = 0.0041                                         avg =        8.0
     overall = 0.0004                                         max =         15
 
                                                Wald chi2(3)      =     195.15
corr(u_i, X)   = 0 (assumed)                    Prob > chi2       =     0.0000
 
                             (Std. Err. adjusted for 11 clusters in SessionID)
------------------------------------------------------------------------------
             |               Robust
LiqPerc1_2_B |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     TreatAD |    7.11397    2.34241     3.04   0.002     2.522932    11.70501
      TreatF |  -4.822081   3.231095    -1.49   0.136    -11.15491    1.510749
     TreatFD |    4.88072   2.068035     2.36   0.018     .8274471    8.933994
       _cons |   49.08114   3.533757    13.89   0.000      42.1551    56.00717
-------------+----------------------------------------------------------------
     sigma_u |  18.226577
     sigma_e |  23.120294
         rho |   .3832775   (fraction of variance due to u_i)
------------------------------------------------------------------------------

I adapted Joseph’s first suggested code to my full sample and corresponding specification, and here are the results. Note that I added the -margins- command at the end to more easily compare to the previous specification. (I think I used it correctly.)

Code:

. /** ANOVA Specification - Adapted from Joseph Coveney's Specification #1 to the Full Sample **/
. anova LiqPerc1_2_B SessionID / SubjectID|SessionID ///
>         Period TreatAD TreatF TreatFD Period#(TreatAD TreatF TreatFD) ///
>         SessionID#Period SessionID#(TreatAD TreatF TreatFD) SessionID#Period#(TreatAD TreatF TreatFD) ///
>         if TreatV==1 & Agent==1, ///
>         repeated(TreatAD TreatF TreatFD Period)
 
                         Number of obs =        528    R-squared     =  0.5415
                         Root MSE      =    23.3386    Adj R-squared =  0.3591
 
                  Source | Partial SS         df         MS        F    Prob>F
 ------------------------+----------------------------------------------------
                   Model |  242543.77        150   1616.9585      2.97  0.0000
                         |
               SessionID |  44843.558         10   4484.3558      1.64  0.1190
     SubjectID|SessionID |  150192.95         55   2730.7808
 ------------------------+----------------------------------------------------
                  Period |  10510.599         14   750.75705      1.38  0.1606
                 TreatAD |  1959.6035          1   1959.6035      3.60  0.0586
                  TreatF |  1229.4479          1   1229.4479      2.26  0.1338
                 TreatFD |  210.05832          1   210.05832      0.39  0.5350
          Period#TreatAD |  5963.2638         11   542.11489      1.00  0.4500
           Period#TreatF |  4479.3611         11   407.21464      0.75  0.6924
          Period#TreatFD |  3958.8969         11   359.89972      0.66  0.7756
        SessionID#Period |   19704.58         35     562.988      1.03  0.4200
       SessionID#TreatAD |          0          0
        SessionID#TreatF |          0          0
       SessionID#TreatFD |          0          0
       SessionID#Period# |
                 TreatAD |          0          0
 SessionID#Period#TreatF |          0          0
       SessionID#Period# |
                 TreatFD |          0          0
                         |
                Residual |  205347.69        377   544.68882
 ------------------------+----------------------------------------------------
                   Total |  447891.46        527   849.88891
 
 
Between-subjects error term:  SubjectID|SessionID
                     Levels:  66        (55 df)
     Lowest b.s.e. variable:  SubjectID
     Covariance pooled over:  SessionID  (for repeated variables)
 
Repeated variable: TreatAD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  1.0000
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
                 TreatAD |      1     3.60   0.0586     .        .      0.0586
       SessionID#TreatAD |      0
                Residual |    377
 -----------------------------------------------------------------------------
 
Repeated variable: TreatF
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  1.0000
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
                  TreatF |      1     2.26   0.1338     .        .      0.1338
        SessionID#TreatF |      0
                Residual |    377
 -----------------------------------------------------------------------------
 
Repeated variable: TreatFD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  1.0000
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
                 TreatFD |      1     0.39   0.5350     .        .      0.5350
       SessionID#TreatFD |      0
                Residual |    377
 -----------------------------------------------------------------------------
 
Repeated variable: Period
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  0.0714
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
                  Period |     14     1.38   0.1606     .        .      0.2507
        SessionID#Period |     35     1.03   0.4200     .        .      0.3832
                Residual |    377
 -----------------------------------------------------------------------------
 
Repeated variables: Period#TreatAD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  0.0909
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
          Period#TreatAD |     11     1.00   0.4500     .        .      0.3254
       SessionID#Period# |
                 TreatAD |      0
                Residual |    377
 -----------------------------------------------------------------------------
 
Repeated variables: Period#TreatF
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  0.0909
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
           Period#TreatF |     11     0.75   0.6924     .        .      0.3932
 SessionID#Period#TreatF |      0
                Residual |    377
 -----------------------------------------------------------------------------
 
Repeated variables: Period#TreatFD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  0.0909
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
 ------------------------+----------------------------------------------------
          Period#TreatFD |     11     0.66   0.7756     .        .      0.4219
       SessionID#Period# |
                 TreatFD |      0
                Residual |    377
 -----------------------------------------------------------------------------
 
. margins, grand
 
Predictive margins                              Number of obs     =        528
 
Expression   : Linear prediction, predict()
 
------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   48.74891   1.015681    48.00   0.000      46.7518    50.74602
------------------------------------------------------------------------------
 
. margins, dydx(TreatAD TreatF TreatFD)
 
Average marginal effects                        Number of obs     =        528
 
Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.TreatAD 1.TreatF 1.TreatFD
 
------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   1.TreatAD |          .  (not estimable)
    1.TreatF |          .  (not estimable)
   1.TreatFD |          .  (not estimable)
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

I don’t think that’s the specification I’m looking for, but it did run. More on the specification selection later, though.

Originally posted by Joseph Coveney View Post

It sounds as if your numerator is the actual outcome variable and the denominator is some kind of adjustment. Have you considered just using the numerator as the outcome variable and putting the denominator in the model as an -offset()-? -xtreg , re- doesn't offer an -offset()- option, but -meglm- does.

Basically, Player 1 is endowed with 100 tokens. His first action is to choose how many of those to keep and how many to “invest” in a “firm” that is managed by Player 2. Player 2 then decides how to split that overall investment between two separate funds, a liquid fund and an illiquid fund. We want to see how this splitting behavior varies across treatments. The relevant literature generally looks for differences in percentages rather than in levels, which is why we use a percentage too. To be precise, Player 1 chooses to invest -TotalInv1_2_A- in the firm. Player 2 splits that into -LiquidInv1_2_B- and -IlliquidInv1_2_B-. Our dependent variable is the percentage of the overall investment that goes into the liquid fund: -LiqPerc1_2_B = 100*LiquidInv1_2_B/TotalInv1_2_A-.

I’ve never used the -offset()- option before, but I did some reading. It may be doable, but I’m not totally sure. I’ll keep digging.

With only certain exceptions, a mixed-model ANOVA requires balanced data for valid test statistics.

I’ve never really used ANOVA before, so I didn’t realize this. Do you know of a good resource where I can find more information on these exceptions? My overall dataset is not balanced, so this could be an issue. (Some experimental sessions were cut short due to time constraints.)

You do understand that that model does not include Period at all, right? Is that your intention?

Yes, I do realize that. The only covariates in my main model are the constant, -TreatAD-, -TreatF-, and -TreatFD-. (Of course the subject-level random effects are also present in the error term, and the error term is also clustered at the session level.) We tried various tests to see if -Period- played a role, and since we found nothing, we cut it from our main specification.

Because if it is, then the ANOVA model corresponding to it is much simplified:

That’s great news to hear! It was the random effects and clustered standard error terms that were tripping me up when “converting” from -xtreg- to -anova-. I didn’t think the specification would be that simple. I adapted your suggested model to my full data:

Code:

. /** ANOVA Specification - Adapted from Joseph Coveney's Specification #2 to the Full Sample **/
. anova LiqPerc1_2_B TreatAD TreatF TreatFD SubjectID if TreatV==1 & Agent==1
 
                         Number of obs =        528    R-squared     =  0.4522
                         Root MSE      =    23.1203    Adj R-squared =  0.3710
 
                  Source | Partial SS         df         MS        F    Prob>F
              -----------+----------------------------------------------------
                   Model |  202533.93         68   2978.4401      5.57  0.0000
                         |
                 TreatAD |  1652.7395          1   1652.7395      3.09  0.0794
                  TreatF |  2008.6687          1   2008.6687      3.76  0.0532
                 TreatFD |  2229.8771          1   2229.8771      4.17  0.0417
               SubjectID |  199712.71         65   3072.5032      5.75  0.0000
                         |
                Residual |  245357.53        459     534.548
              -----------+----------------------------------------------------
                   Total |  447891.46        527   849.88891
 
. margins, grand
 
Predictive margins                              Number of obs     =        528
 
Expression   : Linear prediction, predict()
 
------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   48.74891   1.006182    48.45   0.000     46.77162    50.72621
------------------------------------------------------------------------------
 
. margins, dydx(TreatAD TreatF TreatFD)
 
Average marginal effects                        Number of obs     =        528
 
Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.TreatAD 1.TreatF 1.TreatFD
 
------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   1.TreatAD |   7.388868   4.202126     1.76   0.079    -.8689217    15.64666
    1.TreatF |  -8.295081   4.279173    -1.94   0.053    -16.70428    .1141173
   1.TreatFD |   5.479722   2.682943     2.04   0.042     .2073483     10.7521
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

If this is indeed the correct specification, then all our previous results hold up here. That would be fantastic news (which always makes me doubtful). That leads me to your next point…

It's not clear what issues you're trying to test for, but Session isn't part of your -xtreg , re- model.

Sorry for the confusion. -SessionID- is the level at which the error terms are clustered in my main specification. Does that change the second model you mentioned (just above this?)

Originally posted by Dave Airey View Post

Stata doesn't complain when I run the models with just the two repeated variables, despite the fact that repeated levels are not all measured for most subjects. I note the two within subject factors' levels are not crossed (everyone has an ABA exposure to treatment over periods in SessionID == 10). In the code output of your model and the models below, there is a problem with terms having 0 df which I'm assuming is because we have a subset of the data posted?

Anyway it would be of interest if Matt posted back on application of your syntax to his full dataset and your questions.

The overall panel is still unbalanced, but less so. I ran the -tabulate- command to cross -SessionID- with each of the 4 relevant treatment cells:

Code:

. tabulate SessionID TreatAN if TreatV==1 & Agent==1
 
           |        TreatAN
 SessionID |         0          1 |     Total
-----------+----------------------+----------
         1 |        72          0 |        72
         2 |        75          0 |        75
         3 |        60          0 |        60
         4 |        54         18 |        72
         5 |        96          0 |        96
         6 |        18          0 |        18
         7 |        15          0 |        15
         8 |         0         15 |        15
         9 |        15          0 |        15
        10 |        36         36 |        72
        11 |        63         21 |        84
-----------+----------------------+----------
     Total |       504         90 |       594
 
. tabulate SessionID TreatAD if TreatV==1 & Agent==1
 
           |        TreatAD
 SessionID |         0          1 |     Total
-----------+----------------------+----------
         1 |        72          0 |        72
         2 |        75          0 |        75
         3 |        60          0 |        60
         4 |        72          0 |        72
         5 |        72         24 |        96
         6 |         0         18 |        18
         7 |        15          0 |        15
         8 |        15          0 |        15
         9 |        15          0 |        15
        10 |        54         18 |        72
        11 |        42         42 |        84
-----------+----------------------+----------
     Total |       492        102 |       594
 
. tabulate SessionID TreatFN if TreatV==1 & Agent==1
 
           |        TreatFN
 SessionID |         0          1 |     Total
-----------+----------------------+----------
         1 |        24         48 |        72
         2 |        45         30 |        75
         3 |        30         30 |        60
         4 |        36         36 |        72
         5 |        72         24 |        96
         6 |        18          0 |        18
         7 |        15          0 |        15
         8 |        15          0 |        15
         9 |         0         15 |        15
        10 |        54         18 |        72
        11 |        84          0 |        84
-----------+----------------------+----------
     Total |       393        201 |       594
 
. tabulate SessionID TreatFD if TreatV==1 & Agent==1
 
           |        TreatFD
 SessionID |         0          1 |     Total
-----------+----------------------+----------
         1 |        48         24 |        72
         2 |        30         45 |        75
         3 |        30         30 |        60
         4 |        54         18 |        72
         5 |        48         48 |        96
         6 |        18          0 |        18
         7 |         0         15 |        15
         8 |        15          0 |        15
         9 |        15          0 |        15
        10 |        72          0 |        72
        11 |        63         21 |        84
-----------+----------------------+----------
     Total |       393        201 |       594

I also copy/pasted over from Excel into Notepad and fixed the formatting to fit here in the format that I usually use as my reference guide. (Recall that observations are pairwise, so the number of subjects must be divided by 2 to get the number of observations for each iteration of the game.)

Code:

SessionID   NumSubjects   Games 1-3   Games 4-6   Games 7-9   Games 10-12   Games 13-15   Relevant N (-if TreatV==2-)
1           16            FN-V        FN-V        FD-V                                    72
2           10            FD-V        FD-V        FN-V        FN-V          FD-V          75
3           10            FN-V        FN-V        FD-V        FD-V                        60
4           12            FN-V        FD-V        FN-V        AN-V                        72
5           16            FD-V        FN-V        FD-V        AD-V                        96
6           12            Other       Other       Other       AD-V                        18
7           10            Other       Other       Other       FD-V                        15
8           10            Other       AN-V        Other       Other                       15
9           10            Other       FN-V        Other       Other                       15
10          12            AN-V        AD-V        AN-V        FN-V                        72
11          14            FD-V        AN-V        AD-V        FD-V                        84
Total       132                                                                           594

[End Post 1 of 2]

Last edited by Matt McMahon; 20 Jun 2019, 13:06.

Comment

Matt McMahon

Join Date: Aug 2017
Posts: 11

#11

20 Jun 2019, 13:01

[Post 2 of 2]

Last, I ran your [Dave's] specification, including the -dropemptycells- option. I didn’t follow up with the -SessionID==10- or -SessionID!=10- limiters since they didn’t seem necessary/informative with the full sample, but I’m happy to run those as well if you’d like.

Code:

. /** ANOVA Specification - Adapted from Dave Airey's Specification #1 to the Full Sample **/
. anova LiqPerc1_2_B SubjectID Period / SubjectID#Period TreatAD TreatF TreatFD / SubjectID#(TreatAD TreatF Trea
> tFD) ///
>     Period#(TreatAD TreatF TreatFD) if TreatV==1 & Agent==1, repeated(Period TreatAD TreatF TreatFD) dropempty
> cells
 
                         Number of obs =        528    R-squared     =  1.0000
                         Root MSE      =          0    Adj R-squared =
 
                  Source | Partial SS         df         MS        F    Prob>F
       ------------------+----------------------------------------------------
                   Model |  447891.46        527   849.88891
                         |
               SubjectID |  175384.79         65   2698.2276      5.09  0.0000
                  Period |  8534.1275         14   609.58054      1.15  0.3112
        SubjectID#Period |   235755.9        445   529.78853
       ------------------+----------------------------------------------------
                 TreatAD |  239.51247          1   239.51247
                  TreatF |  93.888875          1   93.888875
                 TreatFD |  69.444455          1   69.444455
       SubjectID#TreatAD |          0          0
       ------------------+----------------------------------------------------
        SubjectID#TreatF |          0          0
       SubjectID#TreatFD |          0          0
          Period#TreatAD |          0          0
           Period#TreatF |          0          0
          Period#TreatFD |          0          0
                         |
                Residual |          0          0
       ------------------+----------------------------------------------------
                   Total |  447891.46        527   849.88891
 
 
Between-subjects error term:  SubjectID
                     Levels:  66        (65 df)
     Lowest b.s.e. variable:  SubjectID
 
Repeated variable: Period
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  0.0714
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
                  Period |     14     1.15   0.3112     .        .      0.2915
        SubjectID#Period |    445
       -----------------------------------------------------------------------
 
Repeated variable: TreatAD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  1.0000
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
                 TreatAD |      1
       SubjectID#TreatAD |      0
       -----------------------------------------------------------------------
 
Repeated variables: Period#TreatAD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =    .
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
          Period#TreatAD |      0
                Residual |      0
       -----------------------------------------------------------------------
 
Repeated variable: TreatF
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  1.0000
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
                  TreatF |      1
       SubjectID#TreatAD |      0
       ------------------+----------------------------------------------------
        SubjectID#TreatF |      0
                Residual |      0
       -----------------------------------------------------------------------
 
Repeated variables: Period#TreatF
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =    .
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
           Period#TreatF |      0
                Residual |      0
       -----------------------------------------------------------------------
 
Repeated variable: TreatFD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =  1.0000
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
                 TreatFD |      1
       SubjectID#TreatAD |      0
       ------------------+----------------------------------------------------
       SubjectID#TreatFD |      0
                Residual |      0
       -----------------------------------------------------------------------
 
Repeated variables: Period#TreatFD
                                          Huynh-Feldt epsilon        =    .
                                          Greenhouse-Geisser epsilon =    .
                                          Box's conservative epsilon =    .
 
                                            ------------ Prob > F ------------
                  Source |     df      F    Regular    H-F      G-G      Box
       ------------------+----------------------------------------------------
          Period#TreatFD |      0
                Residual |      0
       -----------------------------------------------------------------------
 
. margins, grand
 
Predictive margins                              Number of obs     =        528
 
Expression   : Linear prediction, predict()
 
------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   48.74891          .        .       .            .           .
------------------------------------------------------------------------------
 
. margins, dydx(TreatAD TreatF TreatFD)
 
Average marginal effects                        Number of obs     =        528
 
Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.TreatAD 1.TreatF 1.TreatFD
 
------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   1.TreatAD |          .  (not estimable)
    1.TreatF |          .  (not estimable)
   1.TreatFD |          .  (not estimable)
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

I hope this is helpful. At the very least, I’m not getting any errors, though the degrees of freedom is still too small for many of these specifications. But at the end of the day, I’m just looking to match my main specification (with subject-level re and session-level clusters) to an ANOVA counterpart. Thanks again for all your help!

[End Post 2 of 2]

Last edited by Matt McMahon; 20 Jun 2019, 13:07. Reason: Mention that it was Dave's specification (carryover from my previous comment, which was split due to the character length)

Announcement