Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    You can just include the dummies and estimate the model, but that may take a long time.

    Best wishes,

    Joao

    Comment


    • #17
      Thanks. I tried that before but I get an error "too many values" r(134). This is because I'm trying to create 40,000+ dummies (since I have 232x232 countries), so it seems it's too many for Stata to handle.

      Alternatively, I was trying to estimate the model through -xtpqml-, which would automatically include the country-pair dummies (if I understand correctly). However, can I use these coefficients from xtpqml to predict, which is the main aim of the model? The reason why I'm asking this is that the estimation output does not show the coefficients for each of the dummies (it would be such a long list!) and I need these for my prediction, so I want Stata to use them also when calculating the prediction values.

      Regards,

      Ainhoa
      Last edited by Ainhoa Oses; 08 Feb 2019, 05:32.

      Comment


      • #18
        Apologies for getting in touch again, but if you could please take a look at this, I'd really appreciate it. It's quite urgent and I need to clear this up. I noticed my Stata version (SE) does not support more that many variables. I need to come up with a solution for my Gravity Model, where: 1) ppml is used for the estimation; 2) the country-pair FE are taken into account in the estimation; 3) these country-pair dummy coefficients are used in the prediction.

        I was doing something along the lines:
        Code:
        xtpqml Y X1 X2 X3, fe i(countrypair)
        With Y being the stock of migrants for each pair and each year, and the X variables being other variables of interest. I need to use the model for prediction. and in comment #10 you said xtpoisson could not be used for prediction, so I understand that this method xtpqml should not either, since it's just an extension. You said FE would have to be created manually, but I can't do this given the memory issues I explained. What could I do?

        Thanks a lot for all your help.

        Ainhoa
        Last edited by Ainhoa Oses; 08 Feb 2019, 09:30.

        Comment


        • #19
          Dear Ainhoa,

          You can use xtpqml as above and then construct the fixed effects from the results. I believe I have posted the commands to do that some time ago; please search this forum.

          Best wishes,

          Joao

          Comment


          • #20
            Dear Joao,

            Thanks a million for that. I checked and I've made the code below for the prediction (the estimation is as per comment #18 in this thread)

            Code:
            predict fitted,xb
            xi:gen yhat = exp(fitted) 
            egen meany = mean(migra) if yhat !=., by(countrypair)
            egen meanyhat=mean(yhat), by(countrypair)
            gen exp_alpha=meany/meanyhat
            replace yhat = yhat*exp_alpha
            Does this look fine to you? Taking a quick look, the forecasts for the projection horizon seem reasonable.

            Ainhoa

            Comment


            • #21
              Looks good to me :-)

              Comment


              • #22
                Hello there,

                I am trying to estimate a gravity model using -xtpoisson-.

                what is the difference between:

                xtpoisson trade .... , fe r
                and
                xtpoisson trade..., fe

                Which of the two provides better estimations?

                Thank you for your time

                Comment


                • #23
                  Originally posted by Joao Santos Silva View Post
                  Looks good to me :-)
                  Many thanks for that, Joao. I really appreciate it. I have a last-minute urgent doubt, I'd be extremely grateful if you could give me your view. I need to predict migration flows based on the lag of the stocks (in logs) and the same variable but squared (i.e., I have a dynamic forecast). My model is based on xtpqml with country-pair fixed effects, as shown in this thread. My prediction period is time=7 to time=9.

                  I am unsure on how to do this. Because I'm predicting year by year (so that I can then use the predicted value as an input in the following prediction period), I don't know how to get the yhat. What I had done so far is say that yhat is just the mean of what I predicted for that year, which is obviously just the predicted value itself. Is this approach correct? If so, the code I show below should delete all the "replace migrafl = migra_f-migra_f[_n-1]". Keeping this would mean that meany is taking also predicted values, which I don't think is correct. The code below shows migration flows (migrafl) as a function of the log-lag of stocks (llmigra) and this variable squared (llmigrasq). There are more variables that are exogenous but that I've dropped in this post for simplicity. Could you please guide me with the correct approach?

                  Code:
                   xtpqml migrafl llmigra llmigrasq  if migra>migra[_n-1],fe i(countrypair)    
                  * Time = 7  
                  predict fitted,xb
                  xi:gen yhat = exp(fitted)
                  egen meany = mean(migrafl) if yhat !=., by(countrypair)
                  egen meanyhat=mean(yhat), by(countrypair)
                  gen exp_alpha=meany/meanyhat
                  replace yhat = yhat*exp_alpha  
                  gen migra_f = migra if time<=6
                  replace migra_f = yhat+migra[_n-1] if time==7
                  replace migrafl = migra_f-migra_f[_n-1] if time==7  
                  * Time = 8
                  replace llmigra = ln(migra_f[_n-1]) if time==8
                  replace llmigrasq = (ln(migra_f[_n-1]))^2 if time==8  
                  predict fitted1,xb
                  xi:gen yhat1 = exp(fitted1)
                  egen meany1 = mean(migrafl) if yhat1 !=., by(countrypair)
                  egen meanyhat1=mean(yhat1), by(countrypair)
                  gen exp_alpha1=meany1/meanyhat1
                  replace yhat1 = yhat1*exp_alpha1
                  replace migra_f = yhat1+migra_f[_n-1] if time==8
                  replace migrafl = migra_f-migra_f[_n-1] if time==8  
                  * Time = 9
                  replace llmigra = ln(migra_f[_n-1]) if time==9
                  replace llmigrasq = (ln(migra_f[_n-1]))^2 if time==9  
                  predict fitted2,xb
                  xi:gen yhat2 = exp(fitted2)
                  egen meany2 = mean(migrafl) if yhat2 !=., by(countrypair)
                  egen meanyhat2=mean(yhat2), by(countrypair)
                  gen exp_alpha2=meany2/meanyhat2
                  replace yhat2 = yhat2*exp_alpha2
                  replace migra_f = yhat2+migra_f[_n-1] if time==9
                  replace migrafl = migra_f-migra_f[_n-1] if time==9
                  Thanks a million

                  Ainhoa
                  Last edited by Ainhoa Oses; 14 Feb 2019, 12:04.

                  Comment


                  • #24
                    Dear Joao,

                    Apologies for getting in touch again, but I need to clarify this and your advice is really important to me.

                    Thanks a million in advance, and also for all the help you've provided me thus far.

                    Ainhoa

                    Comment


                    • #25
                      Dear Ainhoa,

                      I would estimate the fixed effects just once and then using them for all the predictions (note that these estimates will be noisy and affect the quality of the predictions); to predict for years 8 and 9 you need to use the previous predictions as explanatory variables.

                      Best wishes,

                      Joao

                      Comment


                      • #26
                        Many thanks again, Joao. The problem is that I would have way too many country-pair dummies (more than 40,000) and Stata doesn't have memory to store that many dummies. That's why I used xtpqml using your advice and predicted as shown in this thread. I tought this meant that Stata was taking into account the fixed effects. How can I estimate fixed effects "just once" given this limitation?

                        Just to recap, I have:

                        1) A gravity model that I estimate through PPMl (command xtpqml) with country-pair fixed effects. This is fine
                        2) A dynamic forecast. This is problematic, since I need to find a way to recover the country-pair fixed effects to include in the prediction. I can't create these individually because of memory reasons.

                        Kind regards,

                        Ainhoa

                        Last edited by Ainhoa Oses; 15 Feb 2019, 08:14.

                        Comment


                        • #27
                          Ainhoa,

                          The fixed effects are being estimated as a single variable by observation (exp_alpha), so memory is not a problem.

                          Best wishes,

                          Joao

                          Comment


                          • #28
                            Thanks a lot, Joao, I'm really grateful for your help.

                            Should the "yhat" component of exp_alpha include just the fitted values for the historical years, or should I also include the first prediction year (first, because it's a dynamic forecast and the following year depends on the prediction attained for that previous period)? My understanding is that yhat should only take the fitted values for the historical years. And from then on, I'd use that exp_alpha unchanged for the following projection years, is that correct?

                            Regards.

                            Ainhoa

                            Comment


                            • #29
                              Dear Ainhoa,

                              I no not fully understand your question, but the exp_alpha should indeed be the same for all years.

                              Best wishes,

                              Joao

                              Comment


                              • #30
                                Thanks Joao. Sorry, an example might be easier for me to explain. My historical data for migration is 1960-2010 (every ten years), and my projection period starts in 2020. In order to create exp_alpha in this example, is it correct to say: (i) that meany is the mean of the observed migration for the historical period 1960-2010; and (ii) that meanyhat is the mean of the fitted (predicted) values for the historical period 1960-2010. Just as a side reminder again, my forecast is dynamic.

                                I hope this clarifies it. Thanks a lot for everything.

                                Ainhoa
                                Last edited by Ainhoa Oses; 18 Feb 2019, 03:03.

                                Comment

                                Working...
                                X