Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Interactions in stepwise

    Hi there, I am needing some assistance with adding interaction terms to stepwise regression. I am aware of the limitations of stepwise, but we are needing to conduct this analysis for an assignment. When I use the below code, I get an error saying 'interactions not allowed'. The interactions we have been instructed to use are a mixture of continuous*continuous and categorical*continuous.

    xi: stepwise, pe(0.15) : regress icecreamsold (i.season) (i.year) (i.month) (i.holiday) (i.dayofweek) (i.weekday) (i.weathercat) (temp) (humidity) (windspeed) (i.year#c.temp) (i.weathercat#c.humidity) (humiditywindspeed) (sqtemp) (sqhumidity) sqwindspeed (i.year#c.sqtemp) (i.weathercat#c.sqhumidity)

    Can anyone help me?

    Thanks, Elise

  • #2
    The prefix command xi: is redundant here given your use of factor variable notation and indeed could be very problematic.

    A(n even) bigger deal is where the error message comes from. If it's from stepwise you will have to create the interactions yourself.

    I am glad I don't have to do this assignment. I would just turn in a critique of the method and risk a Fail grade and a formal warning.

    Comment


    • #3
      Thank you for the quick reply. That is exactly what I feel like doing :-)

      In order to generate the variables, do I just do this:

      gen yeartemp = year * temp
      gen weathercathumidity = weathercat * humidity
      gen humiditywindspeed = humidity * windspeed
      gen yearsqtemp = year * sqtemp
      gen weathercatsqhumidity = weathercat * sqhumidity

      And then would the stepwise code look like this:

      stepwise, pe(0.15) : regress icecreamsold (season) (year) (month) (holiday) (dayofweek) (weekday) (weathercat) (temp) (humidity) (windspeed) (yeartemp) (weathercathumidity) (humiditywindspeed) (sqtemp) (sqhumidity) (sqwindspeed) (yearsqtemp) (weathercatsqhumidity)

      My only problem, is that when I input the above code, I just get these results, which do not show individual results for each category in the factor variables:


      Comment


      • #4
        Forward stepwise output attached.
        Attached Files

        Comment


        • #5
          Elise:
          I'm not familiar with -stepwise- (that I consider like a plague), but it may be the case that you have to (re)create the indicators yourself.
          Does the Stata .pdf manual entry give you some clue in this respect?
          As an aside, as per FAQ please post Stata output within CODE delimiters rather than forwarding attachments. Thanks
          Last edited by Carlo Lazzaro; 10 Oct 2016, 08:42.
          Kind regards,
          Carlo
          (Stata 19.0)

          Comment


          • #6
            Indeed. By hiding the fact that some of the variables are interactions, you necessarily lose out on what Stata would do otherwise.

            Comment


            • #7
              Okay great,so I can just report the below output, even though it doesn't show the break down per category?




              Comment


              • #8
                Okay great,so I can just report the attached output, even though it doesn't show the break down per category?
                Attached Files

                Comment


                • #9
                  Elise:
                  please post Stata output within CODE delimiters rather than forwarding attachments. Thanks.
                  Kind regards,
                  Carlo
                  (Stata 19.0)

                  Comment


                  • #10
                    And thanks Carlo, I am having a look now. And sorry about the attachments, I will do so in future.

                    Comment


                    • #11
                      [ stepwise, pe(0.15) : regress icecreamsold (season) (year) (month) (holiday) (dayofwee
                      > k) (weekday) (weathercat) (temp) (humidity) (windspeed) (yeartemp) (weathercathumidit
                      > y) (humiditywindspeed) (sqtemp) (sqhumidity) (sqwindspeed) (yearsqtemp) (weathercatsq
                      > humidity)
                      begin with empty model
                      p = 0.0000 < 0.1500 adding yeartemp
                      p = 0.0000 < 0.1500 adding temp
                      p = 0.0000 < 0.1500 adding sqtemp
                      p = 0.0000 < 0.1500 adding weathercatsqhumidity
                      p = 0.0000 < 0.1500 adding season
                      p = 0.0000 < 0.1500 adding sqwindspeed
                      p = 0.0000 < 0.1500 adding yearsqtemp
                      p = 0.0000 < 0.1500 adding dayofweek
                      p = 0.0004 < 0.1500 adding holiday
                      p = 0.0022 < 0.1500 adding month
                      p = 0.0160 < 0.1500 adding year
                      p = 0.0275 < 0.1500 adding weathercathumidity
                      p = 0.0013 < 0.1500 adding weathercat
                      p = 0.0326 < 0.1500 adding humiditywindspeed
                      p = 0.0228 < 0.1500 adding windspeed
                      p = 0.0453 < 0.1500 adding humidity

                      Source | SS df MS Number of obs = 731
                      -------------+---------------------------------- F(16, 714) = 307.19
                      Model | 2.3921e+09 16 149503157 Prob > F = 0.0000
                      Residual | 347483698 714 486671.845 R-squared = 0.8732
                      -------------+---------------------------------- Adj R-squared = 0.8703
                      Total | 2.7395e+09 730 3752786.59 Root MSE = 697.62

                      --------------------------------------------------------------------------------------
                      icecreamsold | Coef. Std. Err. t P>|t| [95% Conf. Interval]
                      ---------------------+----------------------------------------------------------------
                      yeartemp | 10721.94 1711.716 6.26 0.000 7361.341 14082.54
                      temp | 15668.94 1176.783 13.32 0.000 13358.57 17979.31
                      sqtemp | -11391.09 1191.527 -9.56 0.000 -13730.41 -9051.775
                      weathercatsqhumidity | -3355.503 490.6871 -6.84 0.000 -4318.865 -2392.141
                      season | 446.6095 44.20369 10.10 0.000 359.8247 533.3942
                      sqwindspeed | -12108 3448.986 -3.51 0.000 -18879.37 -5336.637
                      yearsqtemp | -8850.936 1708.824 -5.18 0.000 -12205.86 -5496.015
                      dayofweek | 62.51237 13.03273 4.80 0.000 36.92532 88.09942
                      holiday | -558.5661 156.1316 -3.58 0.000 -865.098 -252.0342
                      month | -40.31794 13.88402 -2.90 0.004 -67.57633 -13.05955
                      year | -944.9592 390.1839 -2.42 0.016 -1711.004 -178.9142
                      weathercathumidity | 3149.895 733.3972 4.29 0.000 1710.022 4589.768
                      weathercat | -947.6953 294.8564 -3.21 0.001 -1526.585 -368.806
                      humiditywindspeed | -8877.876 2402.659 -3.70 0.000 -13595 -4160.754
                      windspeed | 7649.564 2533.722 3.02 0.003 2675.128 12624
                      humidity | 1450.361 723.1273 2.01 0.045 30.65101 2870.071
                      _cons | -2326.184 561.4223 -4.14 0.000 -3428.42 -1223.948
                      --------------------------------------------------------------------------------------

                      .
                      end of do-file

                      ]
                      Last edited by Elise Farley; 10 Oct 2016, 08:58.

                      Comment


                      • #12
                        Elise:
                        set aside that the output you posted was not plugged within CODE delimiters (that is , that you can easily access by clicking on the # button of the Advanced editor) and hence ended up being basically unreadable, the interactions, as they are, seems quite meaningless in practical terms, despite their statistical significance (possibly due to the -stepwise- alchemy).
                        If, for realpolitik reasons, you're forced to follow the painful -stepwise- road, I would suggest to rule out interactions from the set of your predictors.
                        Kind regards,
                        Carlo
                        (Stata 19.0)

                        Comment


                        • #13
                          Apologies (again), I have tried again below.

                          Unfortunately we have to include them as it was requested in the assignment, so should I just report as is, or do I still need to find a way to report the individual levels within categories?

                          Code:
                           stepwise, pe(0.15) : regress icecreamsold (season) (year) (month) (holiday) (dayofwee
                          > k) (weekday) (weathercat) (temp) (humidity) (windspeed) (yeartemp) (weathercathumidit
                          > y) (humiditywindspeed) (sqtemp) (sqhumidity) (sqwindspeed) (yearsqtemp) (weathercatsq
                          > humidity)
                                                begin with empty model
                          p = 0.0000 <  0.1500  adding  yeartemp
                          p = 0.0000 <  0.1500  adding  temp
                          p = 0.0000 <  0.1500  adding  sqtemp
                          p = 0.0000 <  0.1500  adding  weathercatsqhumidity
                          p = 0.0000 <  0.1500  adding  season
                          p = 0.0000 <  0.1500  adding  sqwindspeed
                          p = 0.0000 <  0.1500  adding  yearsqtemp
                          p = 0.0000 <  0.1500  adding  dayofweek
                          p = 0.0004 <  0.1500  adding  holiday
                          p = 0.0022 <  0.1500  adding  month
                          p = 0.0160 <  0.1500  adding  year
                          p = 0.0275 <  0.1500  adding  weathercathumidity
                          p = 0.0013 <  0.1500  adding  weathercat
                          p = 0.0326 <  0.1500  adding  humiditywindspeed
                          p = 0.0228 <  0.1500  adding  windspeed
                          p = 0.0453 <  0.1500  adding  humidity
                          
                                Source |       SS           df       MS      Number of obs   =       731
                          -------------+----------------------------------   F(16, 714)      =    307.19
                                 Model |  2.3921e+09        16   149503157   Prob > F        =    0.0000
                              Residual |   347483698       714  486671.845   R-squared       =    0.8732
                          -------------+----------------------------------   Adj R-squared   =    0.8703
                                 Total |  2.7395e+09       730  3752786.59   Root MSE        =    697.62
                          
                          --------------------------------------------------------------------------------------
                                  icecreamsold |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
                          ---------------------+----------------------------------------------------------------
                                      yeartemp |   10721.94   1711.716     6.26   0.000     7361.341    14082.54
                                          temp |   15668.94   1176.783    13.32   0.000     13358.57    17979.31
                                        sqtemp |  -11391.09   1191.527    -9.56   0.000    -13730.41   -9051.775
                          weathercatsqhumidity |  -3355.503   490.6871    -6.84   0.000    -4318.865   -2392.141
                                        season |   446.6095   44.20369    10.10   0.000     359.8247    533.3942
                                   sqwindspeed |     -12108   3448.986    -3.51   0.000    -18879.37   -5336.637
                                    yearsqtemp |  -8850.936   1708.824    -5.18   0.000    -12205.86   -5496.015
                                     dayofweek |   62.51237   13.03273     4.80   0.000     36.92532    88.09942
                                       holiday |  -558.5661   156.1316    -3.58   0.000     -865.098   -252.0342
                                         month |  -40.31794   13.88402    -2.90   0.004    -67.57633   -13.05955
                                          year |  -944.9592   390.1839    -2.42   0.016    -1711.004   -178.9142
                            weathercathumidity |   3149.895   733.3972     4.29   0.000     1710.022    4589.768
                                    weathercat |  -947.6953   294.8564    -3.21   0.001    -1526.585    -368.806
                             humiditywindspeed |  -8877.876   2402.659    -3.70   0.000       -13595   -4160.754
                                     windspeed |   7649.564   2533.722     3.02   0.003     2675.128       12624
                                      humidity |   1450.361   723.1273     2.01   0.045     30.65101    2870.071
                                         _cons |  -2326.184   561.4223    -4.14   0.000     -3428.42   -1223.948
                          --------------------------------------------------------------------------------------

                          Comment


                          • #14
                            Elise:
                            very well done. Thanks.
                            As far as the substantive matter is concerned, I suspect that, if -stepwise- does not support interactions via -fvvarlist- the output that you posted is the most you can get out of it.
                            If somebody would trust the p-value reported in the outcome tables ought to end up thinking that each and ever predictors is statistically significant: with a bit of low level philosophy, in a world made of statistical significant predictors only, nothing is statistically significant.
                            Kind regards,
                            Carlo
                            (Stata 19.0)

                            Comment


                            • #15
                              Thank you so much for your help Carlo and Nick.

                              I have spent days trying to figure this out, and it has been driving me crazy. So thank you for teaching me, I appreciate it.

                              Comment

                              Working...
                              X