Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by Sebastian Kripfganz View Post
    The p-value range from 0.1 to 0.25 is quite arbitrary. Personally, I would not focus much on this rule of thumb. A high p-value of the Hansen test could indeed be an indication of a too-many-instruments problem, but it could also simply be an indication that there is no evidence to reject the model. Jan Kiviet takes a different stand on these p-values in one of his recent papers:If you ensure from the beginning that the risk of running into a too-many-instruments problem is low, then you would not have to worry much about this rule of thumb.

    There is no general answer whether a p-value between 0.05 and 0.1 for the difference-in-Hansen test is acceptable. If the tested instruments are crucial for the identification of your main coefficients of interest, then this might be worrysome. On the other side, with such a large number of observations I would take much more comfort in such a p-value than with a small sample size, in particular if all other tests are fine.
    Thanks again for replying to my issues. I also compare the number of my instruments and observations with other articles. Comparisons suggest it is quite fine. I believe there is very low probability of instrument proliferation. And thanks for recommending the methodological paper. I will read it.

    Comment


    • #17
      Originally posted by Sebastian Kripfganz View Post
      The p-value range from 0.1 to 0.25 is quite arbitrary. Personally, I would not focus much on this rule of thumb. A high p-value of the Hansen test could indeed be an indication of a too-many-instruments problem, but it could also simply be an indication that there is no evidence to reject the model. Jan Kiviet takes a different stand on these p-values in one of his recent papers:If you ensure from the beginning that the risk of running into a too-many-instruments problem is low, then you would not have to worry much about this rule of thumb.

      There is no general answer whether a p-value between 0.05 and 0.1 for the difference-in-Hansen test is acceptable. If the tested instruments are crucial for the identification of your main coefficients of interest, then this might be worrysome. On the other side, with such a large number of observations I would take much more comfort in such a p-value than with a small sample size, in particular if all other tests are fine.
      Hello, truly sorry for raising up a question again. It's not very critical to my model, but I have been quite confused for a couple of days. In the specification of #14, I also collapse the instruments in the level model except for my core predictor. Because when I collapse all the instruments in both level and transformed models, my core predictor turns to be statistically insignificant. I understand that this is very likely because in large samples, collapsing worsens the statistical efficiency. However, when I switching from the combination of (a b c, lag (2 .) eq(diff) collapse) (a b c, lag (1 1) eq(level) collapse) towards (abc, lag (2 8) eq(diff) collapse) (a b c, lag (1 1) eq(level)), one variable changes from being statistically significant to insignificant and another variable becomes statistically significant. If the changes result from better statistical efficiency, then I think insignificant → significant is reasonable, but significant → insignificant sounds weird to me...

      I personally think that the 2rd version is better because it makes a better trade-off between statistical efficiency and too-many-instrument problem. And also I see you pointed out somewhere that collapsing specific instruments instead of all should be justified with a good reason. But I am not very confident in my understanding and so hope to learn your advice/opinions. Thanks a lot!

      If we just use second and third lags as instruments this leads to the pretty large total of 122 instruments. Using only the second order lags as instruments leads to just 64 instruments and much larger standard errors. When we collapse all the instruments in the standard way 76 instruments remain with results that differ substantially from those that simply skip higher-order lags from the full set of available instruments. Collapsing yields more insignificant regressors too.
      My old codes in #14:
      Code:
      xtabond2 migrate L.migrate a2003 c.co_age##c.co_age dy_schooling marriage hukou_type a2025b InIncome ///
      c.gap_jobdiff3ex##c.gap_jobdiff3ex gap_ppden gap_unemploy gap_enterprise gap_med gap_highedu gap_theater gap_labprod gap_terti gap_LQ19 yr2-yr22, ///
      gmmstyle(migrate, lag(1 1) eq(level) collapse) /// predetermined
      gmmstyle(migrate, lag(2 .) eq(diff) collapse) ///
      gmmstyle(c.gap_jobdiff3ex##c.gap_jobdiff3ex, lag(1 1) eq(level)) ///
      gmmstyle(c.gap_jobdiff3ex##c.gap_jobdiff3ex, lag(2 .) eq(diff) collapse) ///
      gmmstyle(gap_labprod gap_LQ19 gap_terti, lag(1 1) eq(level) collapse) ///
      gmmstyle(gap_labprod gap_LQ19 gap_terti, lag(2 .) eq(diff) collapse) ///
      gmmstyle(gap_ppden gap_enterprise gap_unemploy,lag(0 0) eq(level) collapse) ///
      gmmstyle(gap_ppden gap_enterprise gap_unemploy,lag(1 .) eq(diff) collapse) ///
      ivstyle(gap_highedu gap_med gap_theater, eq(level)) ///
      ivstyle(a2003 co_age dy_schooling marriage hukou_type a2025b InIncome yr2-yr22, eq(level)) ///
      small twostep artests(4) cluster(dest_code)
      New codes:
      Code:
      xtabond2 migrate L.migrate a2003 c.co_age##c.co_age dy_schooling marriage hukou_type a2025b InIncome ///
      c.gap_jobdiff3ex##c.gap_jobdiff3ex gap_ppden gap_unemploy gap_enterprise gap_med gap_highedu gap_theater gap_labprod gap_terti gap_LQ19 yr2-yr22, ///
      gmmstyle(migrate, lag(1 1) eq(level)) /// predetermined
      gmmstyle(migrate, lag(2 8) eq(diff) collapse) ///
      gmmstyle(c.gap_jobdiff3ex##c.gap_jobdiff3ex, lag(1 1) eq(level)) /// endogenous
      gmmstyle(c.gap_jobdiff3ex##c.gap_jobdiff3ex, lag(2 8) eq(diff) collapse) ///
      gmmstyle(gap_labprod gap_LQ19 gap_terti, lag(1 1) eq(level)) /// endogenous
      gmmstyle(gap_labprod gap_LQ19 gap_terti, lag(2 8) eq(diff) collapse) ///
      gmmstyle(gap_ppden gap_enterprise gap_unemploy,lag(0 0) eq(level)) /// not strictly exogenous
      gmmstyle(gap_ppden gap_enterprise gap_unemploy,lag(1 3) eq(diff) collapse) ///
      ivstyle(gap_highedu gap_med gap_theater, eq(level)) /// exogenous
      ivstyle(a2003 co_age dy_schooling marriage hukou_type a2025b InIncome yr2-yr22, eq(level)) ///
      small twostep artests(4) cluster(dest_code)
      Last edited by Huaxin Wanglu; 18 Mar 2021, 21:12.

      Comment


      • #18
        That seems to be a matter about efficiency in the (implicit) first-stage regressions of the regressors on the instruments. These level instruments might be informative for some variables but less informative for others. Adding further informative instruments helps to improve the first-stage fit, while adding further uninformative (weak) instruments worsens the first-stage fit. Adding more (instrumental) variables is not always better, even in large samples.
        https://twitter.com/Kripfganz

        Comment


        • #19
          Originally posted by Sebastian Kripfganz View Post
          That seems to be a matter about efficiency in the (implicit) first-stage regressions of the regressors on the instruments. These level instruments might be informative for some variables but less informative for others. Adding further informative instruments helps to improve the first-stage fit, while adding further uninformative (weak) instruments worsens the first-stage fit. Adding more (instrumental) variables is not always better, even in large samples.
          Thank you for the reply. It helps deepen my understanding. I will choose to report 2rd version in my paper.

          Comment

          Working...
          X