Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Choosing between repeated measures ANOVA and mixed command

    Hello,

    I have experimental data about 99 consumers (called IDconsumer) tasting and evaluating 5 products, on seven consecutive steps. Each consumer notes each of the five products on each of the seven steps, with new information on the products they are provided with between each two steps, potentially changing each consumer notes on the products.
    I’d like to measure the effects of the products, of the steps (i.e. the information given between steps) and of the combined effects products x steps, and I’m a bit confused on the commands to use.

    As observations are not independent (same consumers note each product seven times), I can’t run a simple ANOVA (‘anova note step##product’). But I’m hesitating on using either repeated measures ANOVA or multilevel mixed-effects linear regression. And on both models, I’m hesitating on the form I should put them.
    For the repeated-measures ANOVA, I’ve written it ‘anova note i.step/IDconsumer|step i.product product#step, repeated(product)’ but I’m not sure about the between-subjects error term and nested variables.
    For the mixed model, I’ve written it ‘mixed note i.step##i.product ||IDconsumer:
    Again, as notes are given by the same consumers, I’m wondering if I should add the option for autoregressive residuals…

    Estimation results are the following:
    Model Obs ll(null) ll(model) df AIC BIC
    m_ANOVA 3,465 -9338.748 -8050.892 721 17543.78 21978.27
    m_mixed 3,465 . -8316.189 37 16706.38 16933.94
    pwcompare command gives slightly different results too about the significant differences between steps and products.

    Can you advise me about the most appropriate tests to conduct with my data?

    Thank you very much.

  • #2
    Yann, can you describe a bit more clearly:

    1) What is the dependent variable? It looks like a continuous measure called note. But what does note measure? Is it coded as continuous?

    2) What is a step?

    3) for the syntax of the mixed model you wrote, you're saying the dependent variable is "note." Your independent variables are step, product, and the interaction between those two variables. You are saying you want to treat both as categorical. You also said there are 7 steps and 5 products, so you will get 11 main effects and, I think 6*5= 30 interaction terms. That is a lot of terms. It would be hard to interpret. i think this isn't what you meant to do, but we need a clearer explanation of what you're trying to measure.
    Be aware that it can be very hard to answer a question without sample data. You can use the dataex command for this. Type help dataex at the command line.

    When presenting code or results, please use the code delimiters format them. Use the # button on the formatting toolbar, between the " (double quote) and <> buttons.

    Comment


    • #3
      Originally posted by Yann Raineau View Post
      Each consumer notes each of the five products on each of the seven steps
      It looks as if you have repeated measures for both factors (products and steps), and so the ANOVA model would be something more like
      Code:
      set matsize 11000
      set emptycells drop
      anova note IDconsumer product/IDconsumer|product step/IDconsumer|step product#step, repeated(product step)
      analogous to the example at StataCorp's repeated-measures ANOVA FAQ.

      Either ANOVA with anova or iterative maximum likelihood with mixed would be doable. Keep in mind that the tests reported by Stata after mixed aren't the so-called SAS Type III tests, and so they'll look different from what you get with anova.

      Comment


      • #4
        Thank you for your replies.

        Weiwen, to answer your questions, DV is 'note' which is a continuous variable. Step is {1, 2, ..., 7} and represents each period when people could give a new note to the five products, after comparing them. People were provided new information between each two steps, potentially changing their preferences.
        What I want to measure is actually whether significant differences exist between products at each step and for the same product between different steps. I can start by generating several anovas 'anova note product if step==n' and 'anova note step if product==m' but I would like an overall model taking both factors into account, which seems to me can be modeled with repeated-measures anova or mixed-effects regression.

        Joseph, thanks for the link. I still have some kind of misunderstanding on the way variables must be dealt with. In fact, I can't see the difference between the example you've noticed and the other one on the same page http://www.stata.com/support/faqs/st...nova/#anovat77
        In both cases, subjects face all possible combinations of two variables, being 'dial' and 'period' for the example you've mentioned and 'shape' and 'calib' on the other one. Or am I wrong?
        And the two specifications (being 'anova note factor1/ subject|factor1 factor2 factor1#factor2, repeated(factor2)' and 'anova note subject factor1/ subject|factor1 factor2 / subject|factor2 factor1#factor2, repeated(factor1 factor 2)) give different results in kind of significance of the differences, the latter being less conservative. Which one best estimates the problem?
        What I'm trying to model is that, at each step, one person's note for product A depends both on his previous notes for same product A from previous steps (consistency towards the same product) AND on the notes given for the other products B to E on the current step (products ranking at each step).

        With the mixed model, would you apply some kind of particular specification, for example for the structure of the residual errors (autoregressive structure seemed to work...)?

        Thank you very much again for your help.

        Comment


        • #5
          Just as a side note (hence, being in favor of - mixed - instead of repeated measures ANOVA), beyond three repeated measures sphericity is potentially present.
          Best regards,

          Marcos

          Comment


          • #6
            Originally posted by Yann Raineau View Post
            I still have some kind of misunderstanding on the way variables must be dealt with. In fact, I can't see the difference between the example you've noticed and the other one on the same page http://www.stata.com/support/faqs/st...nova/#anovat77
            In both cases, subjects face all possible combinations of two variables, being 'dial' and 'period' for the example you've mentioned and 'shape' and 'calib' on the other one. Or am I wrong?
            You're wrong. Read the table carefully. It says, "subjects nested under calib", and so subject #1 under calib #1 is not the same person as subject #1 under calib #2.


            Originally posted by Yann Raineau View Post
            And the two specifications (being 'anova note factor1/ subject|factor1 factor2 factor1#factor2, repeated(factor2)' and 'anova note subject factor1/ subject|factor1 factor2 / subject|factor2 factor1#factor2, repeated(factor1 factor 2)) give different results in kind of significance of the differences, the latter being less conservative. Which one best estimates the problem?
            Code:
            anova note factor1/subject|factor1 factor2 factor1#factor2, repeated(factor2)
            is for a different study design (split-plot) from what you have.

            Originally posted by Yann Raineau View Post
            What I'm trying to model is that, at each step, one person's note for product A depends both on his previous notes for same product A from previous steps (consistency towards the same product) AND on the notes given for the other products B to E on the current step (products ranking at each step).
            There are undoubtedly model specifications that could discriminate that all products are evaluated at each step versus individually each through its own seven steps. But I would at least give serious consideration to the model described above (two repeated measures) as a candidate that captures the essential feature of your study's design and yet is not so convoluted as to render postestimation work-up problematic.

            Originally posted by Yann Raineau View Post
            With the mixed model, would you apply some kind of particular specification, for example for the structure of the residual errors (autoregressive structure seemed to work...)?
            If the study was conducted over a short period of time (each consumer's participation lasted only a few hours or so), then I would not automatically reach for autoregressive residual correlation structures. I would first strive for a simpler structure and examine the residuals graphically for gross deviations from assumptions. If you decide to go with anova, then repeated() invokes computation of various so-called correction factors (epsilons) that compensate for lack of sphericity.

            Comment


            • #7
              Thank you very much Joseph for your explanations.
              I'm a bit surprised by the results obtained by model
              Code:
               anova note i.step/IDconsumer|step i.product/IDconsumer|product step#product IDconsumer, repeated(step product)
              which is the one advised by
              HTML Code:
              http://www.stata.com/support/faqs/statistics/repeated-measures-anova/#half713
              Results look really different of the ones obtained by simple anovas.
              For example, for step one
              Code:
              anova note product if step==1
              gives no significant difference between the five products by
              Code:
              pwcompare products, groups tuk
              But with the two-repeated measures model just described, at step one, almost all products are significantly different by
              Code:
              pwcompare product#step, groups tuk
              In fact, two are equal and all other differences are significant.
              Then, I found
              HTML Code:
              http://www.reed.edu/psychology/stata/analyses/parametric/ANOVA/repANOVA.html
              which presents model
              Code:
              anova note step##product ID consumer, repeated(step product)
              which is finally the same model without specifying the error terms. The problem they try to solve looks the same as mine (again, if I'm not wrong!) with same people facing two fixed factors (no between-factor).
              But results from this last model are more consistent with results obtained by ANOVAs step by step or product by product (much less significant differences between products). What is your opinion on this?
              As suggested I've attached residuals graphs obtained with rvfplot command. First one is from model without error terms (model last described) and second graph corresponds to model with error terms, described in the webpage you advised.
              Sorry if you think my questions are a little silly, I'm not very at ease with the error term specification in the anova repeated model!
              Thank you again

              Click image for larger version

Name:	Graphnoerrorterm.png
Views:	1
Size:	68.1 KB
ID:	1370870
              Click image for larger version

Name:	Graphwitherrorterm.png
Views:	1
Size:	60.1 KB
ID:	1370871

              Comment


              • #8
                The difference between the two ANOVA models has been noted on Statalist before. I believe that the psychology community calls the study design "randomized block factorial (RBF)" following Roger E. Kirk's textbook Experimental Design: Procedures for the Behavioral Sciences (various editions and publishers). I call it "strip plot" after the agricultural experimental station archetype. (To be distinguished from the "split plot" design above.)

                The ANOVA model in the Reed College website that you point to is for what Kirk calls "additive RBF" and the one on the StataCorp FAQ is what Kirk calls "non-additive". As David Airey mentioned in the earlier thread, the latter is more general. Unless the product × consumer and step × consumer variance components are known to be negligible (or negative), I recommend avoiding pooling these two variance components in with the residual as a default. You can get estimates of these two model parameters either by setting the mean squares equal to their expectations and solving, or by fitting the same model using mixed and estimating them directly. (See example below.)

                Based upon what you said is your objective ("What I want to measure is actually whether significant differences exist between products at each step and for the same product between different steps."), what's more important than the latter model's generality is that it provides for a more powerful test of the product × step interaction when the variance components are nonnegligible (see example below). This seems to be the case with your study's data inasmuch as you've already observed the difference in power.

                In the illustration below, I first generate artificial data for consumers (cid), products (prd) and steps (stp) with samples size and levels of each factor the same as in your study, and then I subject the artficial data to the "additive RBF" ANOVA model and "nonadditive RBF" (conventional strip-plot) ANOVA model. Finally, I fit the equivalent model using mixed and show how to get the SAS Type III tests of main effects just as what SAS's PROC MIXED reports.

                .ÿversionÿ14.2

                .ÿ
                .ÿclearÿ*

                .ÿsetÿmoreÿoff

                .ÿsetÿseedÿ1370869

                .ÿ
                .ÿquietlyÿsetÿobsÿ99

                .ÿgenerateÿbyteÿcidÿ=ÿ_n

                .ÿgenerateÿdoubleÿcid_uÿ=ÿ4ÿ*ÿrnormal()

                .ÿ
                .ÿquietlyÿexpandÿ5

                .ÿbysortÿcid:ÿgenerateÿbyteÿprdÿ=ÿ_n

                .ÿgenerateÿdoubleÿcXp_uÿ=ÿ2ÿ*ÿrnormal()

                .ÿ
                .ÿquietlyÿexpandÿ7

                .ÿbysortÿcidÿprd:ÿgenerateÿbyteÿstpÿ=ÿ_n

                .ÿ
                .ÿtempfileÿtmpfil0

                .ÿquietlyÿsaveÿ`tmpfil0'

                .ÿ
                .ÿquietlyÿkeepÿifÿprdÿ==ÿ1

                .ÿkeepÿcidÿstp

                .ÿgenerateÿdoubleÿcXs_uÿ=ÿrnormal()

                .ÿ
                .ÿmergeÿ1:mÿcidÿstpÿusingÿ`tmpfil0',ÿassert(match)ÿnogenerateÿnoreport

                .ÿ
                .ÿsortÿcidÿprdÿstp

                .ÿgenerateÿdoubleÿnoteÿ=ÿcid_uÿ+ÿcXp_uÿ+ÿcXs_uÿ+ÿrnormal()ÿ/ÿ2

                .ÿforeachÿvarÿofÿvarlistÿprdÿstpÿ{
                ÿÿ2.ÿÿÿÿÿÿÿÿÿsummarizeÿ`var',ÿmeanonly
                ÿÿ3.ÿÿÿÿÿÿÿÿÿlocalÿ`var'_maxÿ=ÿr(max)
                ÿÿ4.ÿ}

                .ÿquietlyÿreplaceÿnoteÿ=ÿnoteÿ+ÿ0.4ÿifÿprdÿ==ÿ`prd_max'ÿ&ÿstpÿ==ÿ`stp_max'

                .ÿ
                .ÿ*
                .ÿ*ÿBeginÿhere
                .ÿ*
                .ÿ//ÿKirk'sÿ"randomizedÿblockÿfactorial,ÿadditive"
                .ÿanovaÿnoteÿcidÿprd##stp

                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿÿÿÿÿÿ3,465ÿÿÿÿR-squaredÿÿÿÿÿ=ÿÿ0.8164
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿRootÿMSEÿÿÿÿÿÿ=ÿÿÿÿ2.09993ÿÿÿÿAdjÿR-squaredÿ=ÿÿ0.8091

                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿSourceÿ|ÿPartialÿSSÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿÿMSÿÿÿÿÿÿÿÿFÿÿÿÿProb>F
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿModelÿ|ÿÿ65331.383ÿÿÿÿÿÿÿÿ132ÿÿÿ494.93472ÿÿÿÿ112.24ÿÿ0.0000
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿcidÿ|ÿÿ65056.671ÿÿÿÿÿÿÿÿÿ98ÿÿÿ663.84358ÿÿÿÿ150.54ÿÿ0.0000
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿprdÿ|ÿÿ253.60667ÿÿÿÿÿÿÿÿÿÿ4ÿÿÿ63.401667ÿÿÿÿÿ14.38ÿÿ0.0000
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿstpÿ|ÿÿ8.8291315ÿÿÿÿÿÿÿÿÿÿ6ÿÿÿ1.4715219ÿÿÿÿÿÿ0.33ÿÿ0.9194
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿprd#stpÿ|ÿÿ12.276085ÿÿÿÿÿÿÿÿÿ24ÿÿÿ.51150356ÿÿÿÿÿÿ0.12ÿÿ1.0000
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿResidualÿ|ÿÿ14693.104ÿÿÿÿÿÿ3,332ÿÿÿ4.4096951ÿÿ
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿTotalÿ|ÿÿ80024.487ÿÿÿÿÿÿ3,464ÿÿÿ23.101757ÿÿ

                .ÿ
                .ÿ//ÿKirk'sÿ"randomizedÿblockÿfactorial,ÿnonadditve"ÿ(strip-plot)
                .ÿsetÿmatsizeÿ11000

                .ÿanovaÿnoteÿcidÿprd/prd|cidÿstp/stp|cidÿprd#stp

                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿÿÿÿÿÿ3,465ÿÿÿÿR-squaredÿÿÿÿÿ=ÿÿ0.9927
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿRootÿMSEÿÿÿÿÿÿ=ÿÿÿÿ.496784ÿÿÿÿAdjÿR-squaredÿ=ÿÿ0.9893

                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿSourceÿ|ÿPartialÿSSÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿÿMSÿÿÿÿÿÿÿÿFÿÿÿÿProb>F
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿModelÿ|ÿÿ79444.026ÿÿÿÿÿÿ1,112ÿÿÿ71.442469ÿÿÿÿ289.48ÿÿ0.0000
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿcidÿ|ÿÿ65056.671ÿÿÿÿÿÿÿÿÿ98ÿÿÿ663.84358ÿÿÿÿÿ24.28ÿÿ0.0000
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿprdÿ|ÿÿ253.60667ÿÿÿÿÿÿÿÿÿÿ4ÿÿÿ63.401667ÿÿÿÿÿÿ2.32ÿÿ0.0565
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿprd|cidÿ|ÿÿ10719.944ÿÿÿÿÿÿÿÿ392ÿÿÿ27.346796ÿÿ
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿstpÿ|ÿÿ8.8291315ÿÿÿÿÿÿÿÿÿÿ6ÿÿÿ1.4715219ÿÿÿÿÿÿ0.26ÿÿ0.9572
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿstp|cidÿ|ÿÿ3392.6989ÿÿÿÿÿÿÿÿ588ÿÿÿÿ5.769896ÿÿ
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿprd#stpÿ|ÿÿ12.276085ÿÿÿÿÿÿÿÿÿ24ÿÿÿ.51150356ÿÿÿÿÿÿ2.07ÿÿ0.0017
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿResidualÿ|ÿÿ580.46108ÿÿÿÿÿÿ2,352ÿÿÿ.24679468ÿÿ
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿTotalÿ|ÿÿ80024.487ÿÿÿÿÿÿ3,464ÿÿÿ23.101757ÿÿ

                .ÿ
                .ÿ//ÿiterativeÿmaximum-likelihoodÿmethod
                .ÿmixedÿnoteÿi.prd##i.stpÿ||ÿcid:ÿ||ÿcid:ÿR.prdÿ||ÿcid:ÿR.stpÿ,ÿ///
                >ÿÿÿÿÿÿÿÿÿremlÿnolrtestÿnofetableÿnolog

                Mixed-effectsÿREMLÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿÿ=ÿÿÿÿÿÿ3,465
                Groupÿvariable:ÿcidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿÿ=ÿÿÿÿÿÿÿÿÿ99

                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿÿÿÿÿ35
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿÿÿÿÿ35.0
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿÿÿÿÿ35

                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(34)ÿÿÿÿÿ=ÿÿÿÿÿÿ60.55
                Logÿrestricted-likelihoodÿ=ÿ-4784.0863ÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿÿ=ÿÿÿÿÿ0.0034

                ------------------------------------------------------------------------------
                ÿÿRandom-effectsÿParametersÿÿ|ÿÿÿEstimateÿÿÿStd.ÿErr.ÿÿÿÿÿ[95%ÿConf.ÿInterval]
                -----------------------------+------------------------------------------------
                cid:ÿIdentityÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(_cons)ÿ|ÿÿÿ18.02784ÿÿÿ2.710163ÿÿÿÿÿÿ13.42706ÿÿÿÿ24.20509
                -----------------------------+------------------------------------------------
                cid:ÿIdentityÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(R.prd)ÿ|ÿÿÿ3.871434ÿÿÿ.2790514ÿÿÿÿÿÿÿ3.36138ÿÿÿÿ4.458883
                -----------------------------+------------------------------------------------
                cid:ÿIdentityÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(R.stp)ÿ|ÿÿÿ1.104622ÿÿÿ.0673169ÿÿÿÿÿÿ.9802577ÿÿÿÿ1.244763
                -----------------------------+------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(Residual)ÿ|ÿÿÿ.2467945ÿÿÿ.0071967ÿÿÿÿÿÿ.2330848ÿÿÿÿ.2613106
                ------------------------------------------------------------------------------

                .ÿ/*ÿMoreÿefficientÿalternativeÿequivalentÿspecification:
                >ÿmixedÿnoteÿi.prd##i.stpÿ||ÿcid:ÿR.prd,ÿcovariance(exchangeable)ÿ||ÿstp:ÿ,ÿ///
                >ÿÿÿÿÿÿÿÿÿremlÿnolrtestÿnofetableÿnologÿ*/
                .ÿquietlyÿestatÿdf,ÿmethod(satterthwaite)ÿpost

                .ÿcontrastÿprd#stp,ÿsmall

                Contrastsÿofÿmarginalÿlinearÿpredictions

                Marginsÿÿÿÿÿÿ:ÿasbalanced

                -----------------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿddfÿÿÿÿÿÿÿÿÿÿÿFÿÿÿÿÿÿÿÿP>F
                -------------+---------------------------------------------
                noteÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿprd#stpÿ|ÿÿÿÿÿÿÿÿÿ24ÿÿÿÿ2352.00ÿÿÿÿÿÿÿÿ2.07ÿÿÿÿÿ0.0017
                -----------------------------------------------------------

                .ÿ//ÿSASÿTypeÿIIIÿtestsÿofÿmainÿeffects
                .ÿcontrastÿprd,ÿsmall

                Contrastsÿofÿmarginalÿlinearÿpredictions

                Marginsÿÿÿÿÿÿ:ÿasbalanced

                -----------------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿddfÿÿÿÿÿÿÿÿÿÿÿFÿÿÿÿÿÿÿÿP>F
                -------------+---------------------------------------------
                noteÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿprdÿ|ÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿ392.00ÿÿÿÿÿÿÿÿ2.32ÿÿÿÿÿ0.0565
                -----------------------------------------------------------

                .ÿcontrastÿstp,ÿsmall

                Contrastsÿofÿmarginalÿlinearÿpredictions

                Marginsÿÿÿÿÿÿ:ÿasbalanced

                -----------------------------------------------------------
                ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿddfÿÿÿÿÿÿÿÿÿÿÿFÿÿÿÿÿÿÿÿP>F
                -------------+---------------------------------------------
                noteÿÿÿÿÿÿÿÿÿ|
                ÿÿÿÿÿÿÿÿÿstpÿ|ÿÿÿÿÿÿÿÿÿÿ6ÿÿÿÿÿ588.00ÿÿÿÿÿÿÿÿ0.26ÿÿÿÿÿ0.9572
                -----------------------------------------------------------

                .ÿ
                .ÿexit

                endÿofÿdo-file


                .

                Comment

                Working...
                X