Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Margins(plot) of treatment effect, rather than Y, for values of a covariate

    I'm running a multivariate regression (outcome variable is continuous, happens to be GPA). The covariate of interest is a dummy variable for treatment status; another of the covariates is a pre-score. We want to look at how the treatment effect differs at various values of pre-score. The structure of the model is not complicated:


    regress GPA treatment pre_score X3 X4 X5...

    What I want is a graph that shows what the treatment effect is (values of Beta1) at various values of pre-score (X2). It's straightforward to get a graph with values of the OUTCOME at various values of X2:
    margins, at(pre_score= (1(0.25)5)) post
    marginsplot

    I have consulted an array of resources and tried alternatives using marginscontplot, coefplot with recast, the dy/dx option, and so forth. I remain unsuccessful. But this seems like something that there must be a way to do; wanting to know if a treatment effect varies for values of a control (say, income) must be common.

    Can anyone direct me to the right command, or options for Margins, to output values of Beta1 (coefficient on treatment dummy), rather than of Y (GPA), at values of the pre_score?

    Thank you very much.

  • #2
    Your model is a flat linear model with no interaction terms. So the treatment effect is just the coefficient of treatment at all values of X2. There is nothing more to it, and there is nothing to graph here. If you did graph it, it would just be a horizontal line at height = coefficient of treatment.

    Comment


    • #3
      I've already done checks by hand, and the value of the treatment coefficient does differ based on values of prescore. The treatment effect is lower at the tails of the prescore, and larger in the middle of the pre-score range. It's possible (likely) that your response indicates that I haven't made myself clear in some fashion; but a graph where the X axis was pre-score and the Y axis was treatment coefficient at that value of pre-score would show a curve of differing values of beta 1.
      Is what I'm hoping for clearer, or do I need to try again? [thank you for your patience, as this is my first time asking a STATA question online rather than in person]

      Comment


      • #4
        I think you need to show the exact commands you ran, the output of the regression, and the output of margins. If you can save the graph as a .png and attach it as well, that would be good. There is something you did that differs from what you said. If you found a different effect of treatment at different levels of pre_score from the regression command you show in #1, then you did something wrong, because it is mathematically impossible.

        I don't have your data set at hand, but look at the following code, which uses the built-in auto.dta, and is exactly analogous to what you described in #1:

        Code:
        sysuse auto, clear
        
        regress price i.foreign headroom length trunk weight
        
        margins, dydx(foreign) at(headroom = (2(1)5))
        marginsplot

        Comment


        • #5
          If I just run the regression on restricted samples (say, students with pre-scores 1-1.5, then >1.5-2, etc) the results look roughly like:
          Pre-Score Effect (B1)
          1-1.5 -.14
          1.5-2 -.08
          2-2.5 .05
          2.5-3 .16
          3-3.5 .12
          3.5-4 .19
          4-4.5 .10
          4.5-5 .02
          I'm hoping to end up producing a graph that illustrates those differing treatment effects for students at different points in the pre_score distribution.



          First attempt of
          Code:
          set more off
          regress F1_Lt_Grd_Pt_Avg mindset mind_pre p_tot_response female FirstGen hsgpa_4 foreign
          margins, dydx(mindset) at(mind_pre = (1(0.1)5))
          marginsplot, noci
          produces what I have attached as 1st Output.

          Following the regression syntax with, instead
          Code:
          eststo SaturationTest: regress F1_Lt_Grd_Pt_Avg mindset mind_pre p_tot_response female FirstGen hsgpa_4 foreign
          margins, at(mind_pre = (1(0.33)5)) post
          coefplot SaturationTest, at ytitle(Mindset Treatment Coef) xtitle(Mindset Pre-Score) ///
          recast(line) lwidth(*2) ciopts(recast(rline) lpattern(dash))
          produces what I have attached as 2nd Output.

          Another pair of attempts are closer in a sense; they show for GPA values what I want for treatment effect. This syntax:
          Code:
          regress F1_Lt_Grd_Pt_Avg mindset mind_pre p_tot_response female FirstGen hsgpa_4 act_eq foreign
          eststo SaturationTest: margins, at(mind_pre = (1(0.25)5)) post
          marginsplot
          produces 3rd Output, and the closest analogue of what I want is

          Code:
          regress F1_Lt_Grd_Pt_Avg mindset mind_pre p_tot_response female FirstGen hsgpa_4 act_eq foreign
          eststo SaturationTest: margins, at(mind_pre = (1(0.25)5)) post
          coefplot SaturationTest, at ytitle(Mindset Treatment Coef) xtitle(Mindset Pre-Score) ///
          recast(line) lwidth(*2) ciopts(recast(rline) lpattern(dash))
          which is attached as Output 4. But again, that shows a Y-axis of the outcome value (GPA). I want a Y-axis of the differences in treatment effect.


          I hope this is more helpful.
          Attached Files
          Last edited by Daniel Fitzpatrick; 18 Jan 2017, 16:38. Reason: correcting code delimiters

          Comment


          • #6
            OK. Much clearer. What you show in your first table suggests that, at least when you do separate regressions, you get different effects of pre-score.Moreover, they don't just bounce around: there seems to be a fairly consistent trend that the effect of treatment increases with increases in pre-score.

            Now, after that table you post a bunch of commands and some graphs that I have difficulty relating to this because the variables involved are no longer called GPA, treatment, pre-score, as mentioned originally. Nevertheless, I will speculate that F1_Lt_Grd_Pt_Avg corresponds to GPA, mindset corresponds to treatment and mind_pre corresponds to pre-score. Since I began thinking about this in terms of GPA, treatment, and pre-score, and since those are easier to type, I will continue using those in the rest of this response.

            Your regressions are simply the wrong model for capturing the varying effect of treatment that you observed. In order for a regression model to express a varying effect of treatment depending on the value of pre-score it must include a treatment X pre-score interaction term. Thus your regression should be:

            Code:
            regress GPA i.treatment##c.pre_score // AND YOUR OTHER VARIABLES
            If you follow that with
            Code:
            margins, dydx(treatment) at(pre_score = (1(0.1)5))
            marginsplot
            you will see this relationship. The non-interaction linear model is mathematically incapable of doing this.

            Comment


            • #7
              That got me exactly what I need. Thank you very much--particularly for being polite/patient with my first post not providing enough information.

              This is precisely why I wanted to consult someone knowledgeable--I thought that I might need to change the way I was going about producing these results, but wasn't sure of the correct approach for doing so, or where to find it. Thank you!

              Comment

              Working...
              X