Use of Time Operators with Zero-Inflated Poisson Regression

[Nihat Mugurtay]

Dear all,

I have a count-dependent variable and my main independent variable is also a count variable with many zeros. Therefore, I am trying to use a zero-inflated Poisson regression model in STATA 17.

My data covers the years between 2014 and 2020. I want to build a pooled time series model using a zero-inflated Poisson estimator, which STATA has a "zip" command for this.

When I use the zip command with lagged variables, STATA throws an error, warning me not to use time-series operates such "L.". I wrote the syntax below. When I exclude "L." operator, I can run the command; however, my model affects autocorrelation and endogeneity without lagging. I can create different lagged variables without using time operators, but I don't understand why STATA does not allow to use of time operators with the "zip" command? What is the reason for this?

Time-Series Pooled Model with Year-Fixed Effects. The Code that STATA 17 gives error for using time operators.

zip  logqataraidint L.logqatariaidint c.L.loguae i.muslim  i.arableague   L.cce L.logdacoda L.logqatarexports L.logqatarimports L.loggdpper L.infant L.logpopulation L.logaffected i.year, inflate(qatarbilateral) vce(cluster iso)

When I omit time operators, I can run the command above.

Best,

[Rich Goldstein]

did you -tsset- your data first?

[Nihat Mugurtay]

Dear Rich Goldstein ,

Yes, I did

tsset iso year

I fixed the dependent variable, the estimation l I want to run is here:

zip  qataraidint L.qataraidint c.L.loguae i.muslim  i.arableague  L.cce L.logdacoda L.logqatarexports L.logqatarimports L.loggdpper L.infant L.logpopulation L.logaffected i.year, inflate(qatarbilateral) vce(cluster iso)

This code below is working:

poisson  qataraidint L.qataraidint c.L.loguae i.muslim  i.arableague  L.cce L.logdacoda L.logqatarexports L.logqatarimports L.loggdpper L.infant L.logpopulation L.logaffected i.year, vce(cluster iso)

[Nihat Mugurtay]

Below, I provide a part of the dataset.

* Example generated by -dataex-. For more info, type help dataex
clear
input byte qataraidinteractions float(logqatarbilateral loguae) byte muslim float(cce logdacoda logqatarexports logqatarimports loggdpper) double infant float(logpopulation logaffecteddisasters)
.         0   1.94591 1   -1.29538  9.547955         0         0         .  90.5 16.849499 14.764163
.         0   6.75227 1          . 10.891335         0         0         .  87.9 16.888527 12.229277
.         0  8.064636 1  -1.263366  11.88295         0         0  9.903008  85.3 16.933495 12.656096
.         0  5.940171 1  -1.351042 11.975697         0         0  9.940962  82.7 16.980179  8.466742
.         0  6.893656 1 -1.3452814 12.251638         0         0  9.911789    80 17.023394   8.61975
.         0  7.724005 1 -1.4436092  12.48776         0         0  9.981389  77.3  17.06022 10.708176
.         0  6.484635 1 -1.4328898  12.55314         0         0 10.003672  74.6 17.090126 14.619264
.         0 11.539266 1 -1.5873314  12.70737         0         0 10.108237  71.9 17.115065 10.317417
.         0   7.16858 1 -1.6382866 12.968135         0         0 10.124054  69.2 17.137747 13.023568
.         0  8.465268 1  -1.534796 13.241676         0         0 10.293926  66.7 17.161716  11.09524
.         0  7.672292 1  -1.636177  13.28729         0         0 10.400663  64.2 17.189182 10.740735
.         0  8.192571 1  -1.579174 13.309282         0         0 10.373487  61.8 17.220613 14.382422
.         0  9.146655 1 -1.4197414 13.243042         0         0 10.459434  59.5  17.25469 10.849357
.         0  8.654169 1 -1.4365103  12.96427         0         0 10.478983  57.3 17.289637  9.981744
0         0  8.967887 1  -1.354829 12.910454 11.111816  9.552298 10.472308  55.2 17.323193  11.91929
0   1.94591  8.705166 1  -1.342216 12.854314  12.47501  9.974607 10.455946  53.2 17.353962 11.774128
3         0  8.397509 1 -1.5261716 12.726452  10.91769         0 10.450518  51.3 17.381742         0
0  .6931472  8.036573 1 -1.5156256 12.603774 10.223567 10.182597 10.451164  49.5  17.40722  9.327234
2 1.7917595  7.053586 1  -1.487624 12.500885  7.702729         0 10.439144  47.9 17.431065 16.418497
2 2.0794415  6.206576 1  -1.400733 12.640423 10.369761  8.524943 10.454383  46.4 17.454195  11.78251
2  2.484907  6.182085 1 -1.4754045  12.36706  8.643354 10.216035 10.407554    45 17.477234 10.855474
.         0 4.0775375 1  -.8572564 10.005006         0         0 12.359983  24.1 14.943367         0
.         0 4.1271343 1          .  10.06896         0         0  12.44904  22.9 14.933982         0
.         0 4.0943446 1   -.868602 10.182482         0         0 12.496407  21.7 14.930984 12.165823
.         0   4.59512 1  -.8122751 10.275602         0         0  12.55396  20.4  14.92724         0
.         0 4.2904596 1  -.6991984  9.926325         0         0  12.61182  19.1 14.923062  7.824046
.         0 4.6821313 1  -.7861931  9.931297         0         0 12.670728  17.8 14.917945  12.90047
.         0  5.003946 1   -.803845  9.985575         0         0 12.734387  16.6 14.911635         0
.         0   4.94876 1   -.687789  9.934259         0         0 12.800056  15.3 14.904078  5.416101
.         0  5.170484 1  -.5937066 10.122262         0         0  12.88005  14.1 14.896405         0
.         0  4.406719 1  -.5384951 10.065862         0         0 12.919782  12.9 14.889666  8.817298
.         0   2.70805 1  -.5252154  9.995975         0         0 12.961145  11.8   14.8847  9.546813
.         0         0 1  -.6829866  9.831185         0         0  12.98897  10.8  14.88201         0
.         0  7.459339 1  -.7264909  9.623112         0         0 13.004695    10  14.88036 12.345835
.         0  7.900266 1  -.6984319  9.342333         0         0 13.016498   9.3 14.878528         0
0  .6931472  6.329721 1 -.54816353  9.604609 13.094913 12.604257 13.036157   8.8 14.876457         0
2 1.3862944  6.383507 1  -.4790351  9.794788  12.59088  12.31258 13.061013   8.5 14.873545 10.652542
3         0  6.453625 1  -.4051488  9.257224 12.497812 11.981375 13.095224   8.4 14.871946  8.411833
3         0  6.188264 1  -.4207218  9.459541 11.869053  12.18207 13.133462   8.4 14.871026  9.952373
1 2.0794415  3.295837 1  -.4786321  9.605688 12.406218  12.69799 13.175336   8.5  14.86856  6.684612
3  .6931472  4.406719 1  -.5330888  9.591854  12.12688  10.43793  13.20051   8.6   14.8643 12.244452
6 1.7917595         0 1 -.54019606  9.723344   12.2939  9.422848 13.165932   8.8  14.85852         0
.         0  7.957877 1  -.9379111  9.318747         0         0 12.725495  33.9  17.25086  5.370638
.         0  7.667158 1          .  9.278466         0         0 12.741956  33.3 17.263958 10.723774
.         0         0 1  -.8756811  9.856396         0         0 12.783693  32.4 17.276709  7.742836
.         0         0 1  -.6922612  9.982853         0         0  12.84046  31.3 17.289467 12.257293
.         0         0 1   -.679478  10.19242         0         0 12.869383  30.1 17.302645 10.235665
.         0  6.838405 1   -.482172 10.297116         0         0 12.912808  28.8 17.316545  7.509335
.         0  8.257645 1  -.5213351  9.992871         0         0 12.914954  27.6 17.331257 11.007402
.         0  7.023759 1  -.5599289 10.237743         0         0 12.932876  26.4  17.34677  7.173958
.         0 3.9512436 1  -.5945694  10.00066         0         0  12.94023  25.3 17.363132  10.98614
.         0  6.943122 1  -.5778834  9.835262         0         0 12.938883  24.4 17.380352  7.843064
.         0         0 1  -.5249873  9.535462         0         0   12.9562  23.6 17.398403         0
.         0         0 1  -.5445604  9.303284         0         0 12.965954  22.9 17.417236  6.675823
.         0         0 1  -.5032518  9.184817         0         0 12.979874  22.5  17.43675 4.6151204
.         0  8.342363 1   -.473337  9.151015         0         0 12.987462  22.1 17.456778         0
0 1.3862944  6.079933 1  -.5999988 9.1560955  15.83821  11.00411 13.004422  21.9 17.477114  6.040255
0   2.70805         0 1  -.6456115  8.615408 15.342526  12.16466   13.0203  21.6 17.497568 10.463103
0 1.7917595         0 1  -.6777601  9.206132  15.11274  12.19477 13.031285  21.3 17.518082         0
0 1.7917595         0 1  -.6018301  9.358847 15.249723 12.852998 13.023752  20.9  17.53853  11.73607
0 1.7917595         0 1    -.65687  9.101529 16.041676 14.288182  13.01462  20.4 17.558603  5.298317
1  1.609438         0 1  -.6211485  9.392412 15.301207 12.924196  13.00523    20 17.577944  11.73627
1  .6931472  1.609438 1  -.6418455  9.339173  15.02096  5.327502 12.934517  19.5  17.59631  9.615806
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         . 1.0986123
.         0         0 .          .         .         0         0         .     .         . 10.045855
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .  7.824046
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 .          .         .         0         0         .     .         .         0
.         0         0 0  -1.522685 10.313376         0         0 10.441825 121.5 16.612516 11.278822
.         0         0 0          .  10.24217         0         0 10.450012 118.2 16.645527  10.60542
.         0         0 0 -1.1836292 10.632388         0         0  10.54481 114.5  16.67882         0
.         0         0 0 -1.3206655 10.785208         0         0 10.540483 110.4 16.712608  6.715384
.         0 1.7917595 0 -1.3135588  11.70377         0         0  10.60989 105.7 16.747139 12.790367
.         0 1.0986123 0  -1.296139 10.253088         0         0 10.714526 100.9 16.782515 9.2103405
.         0  .6931472 0 -1.2200612  8.669056         0         0 10.787612  95.7  16.81871 10.964658
.         0         0 0 -1.2896932  9.299084         0         0 10.881923  90.6 16.855516 11.071983
.         0         0 0 -1.2846206  9.968619         0         0 10.950672  85.5 16.892622  11.39521
.         0         0 0 -1.4046545  9.612132         0         0 10.922185  80.6  16.92966  12.43322
.         0         0 0 -1.3260952  9.642901         0         0 10.928567  75.9 16.966375 12.153626
.         0 1.0986123 0 -1.3442773  9.399472         0         0 10.926356  71.5 17.002716 11.415136
.         0         0 0   -1.26847  9.517752         0         0 10.972347  67.4 17.038694 14.421955
.         0         0 0 -1.3094634  9.564793         0         0 10.985185  63.7 17.074215  6.907755
0         0         0 0 -1.4439437  9.132595 14.821192  9.121247  10.99731  60.5 17.109188         0
0         0         0 0 -1.3950136 10.353448 15.124423   7.31402 10.972312  57.7 17.143578 9.1694145
end

------------------ copy up to and including the previous line ------------------

Listed 100 out of 4788 observations
Use the count() option to list more



What I want to run:

zip  qataraidinteractions L.qataraidinteractions c.L.loguae i.muslim  i.arableague L.cce L.logdacoda L.logqatarexports L.logqatarimports L.loggdpper L.infant L.logpopulation L.logaffected i.year, inflate(qatarbilateral) vce(cluster iso)

[Joao Santos Silva]

Dear Nihat Mugurtay,

My suggestion is that you abandon the ZIP model and just use Poisson. You did not tell us what is the variable you are modelling, but nothing that you say suggests that a ZIP would be adequate. Also, please note that a Poisson regression may be adequate even if most of the data are zeros.

Best wishes,

Joao

[Nihat Mugurtay]

Dear Joao Santos Silva,

Thank you for your suggestion. Actually, my dependent variable is the count of Qatar's aid operations between 2014-2020. The main independent variable is the total count of Qatar's global diplomatic interactions. Therefore, both variables are count variables with many zeros. That's why I tried to use a zero-inflated poisson model. If I adopt a Poisson regression, my pooled time-series model with year-fixed effects will be:

poisson  qataraidinteractions L.qataraidinteractions qatarbilateral c.L.loguae i.muslim  i.arableague L.cce L.logdacoda L.logqatarexports L.logqatarimports L.loggdpper L.infant L.logpopulation L.logaffected i.year,  vce(cluster iso)

[Weiwen Ng]

Dear Nihat Mugurtay,

My suggestion is that you abandon the ZIP model and just use Poisson. You did not tell us what is the variable you are modelling, but nothing that you say suggests that a ZIP would be adequate. Also, please note that a Poisson regression may be adequate even if most of the data are zeros.

Best wishes,

Joao

I'd like to amplify what Joao said.

A lot of users tab their dependent variable, see what they think is a lot of zeroes, and think that maybe they should use a ZIP or ZINB model.

I'd encourage analysts to consider this process instead. A zero inflated model assumes that there's one fraction of respondents who always respond zero, while the rest have a Poisson/neg. binomial distribution whose rate is influenced by XB and which can also produce zeroes.

In this context, each obs is a country-year (and there's only one country). The DV is global diplomatic interactions. In this case, you would be assuming that in some fraction of years, Qatar's expected number of global diplomatic interactions is exactly zero. (This seems like a bit of a strong assumption to me.) In other years, it would follow a count model with some mean, but remember that a Poisson distribution with lambda = 1 will produce a zero about 1/3 of the time.

If you can articulate why that might happen, however, then sure, a zero-inflated model would be justified to try. It might not be necessary, but you wouldn't be wrong to fit one.

Consider that when we are learning statistics, a lot of us wind up checking if the marginal distribution of our DV is normal-ish. If it looks wrong, we get worried. When we do this, we are actually incorrect. OLS requires that the residuals should be normally distributed to be efficient. If the residual distribution isn't normal, you can fix the problem with another type of regression, or you can accept that OLS won't be efficient but should still be unbiased. Similarly, seeing a lot of zeroes in the marginal (i.e. unconditional) distribution of your DV isn't an issue per se.

Also worth considering: assume that the Poisson model is adequate but the ZIP model is superior. I still think there's a case to consider the Poisson model over the ZIP model unless your important coefficients have markedly different effects. This is because it's hard to explain the ZIP model to people unfamiliar with it. Say I gave you a ZIP model output table. Do you even know what the coefficients under the zero-inflated part mean?

In plain English, they are actually log odds, as if you had fit a logistic model for being a structural zero. There, now you understand. But again, this isn't something that's that easy to comprehend if you aren't used to the setup.

[Jeff Wooldridge]

When applied to a different data set Stata says zip doesn’t allow time series operators. I’m not sure why. You can create the lags yourself.

Agree that Poisson should do what you want. Almost always interest is in the effects on the mean.

[Nihat Mugurtay]

Dear Jeff Wooldridge and Weiwen Ng,

Thank you for your valuable suggestions,
Now, I understand when we should use zip.
I will use Poisson regression (pooled time series as below). I will use bilateral diplomatic interactions without logging.

 poisson  qataraidint L.qataraidint L.qatarbilateral c.L.loguae i.muslim  i.arableague  L.cce L.logdacoda L.logqatarexports L.logqatarimports L.loggdpper L.infant L.logpopulation L.logaffected i.year, vce(cluster iso)

Best,