time series probit regression analysis

[Philip Murphy]

Hi,

I am attempting to use STATA to run a probit regression on time series using the xtprobit function however I am not getting the expected results.

I am trying to measure what causes football clubs to go insolvent I am regressing insolvency events against residuals from a fixed effects regression which represent shocks to a clubs league position caused by factors other than wages and another set of residuals which represent shocks to clubs revenue caused by factors other than league position.
I am also using the division in which the club were in at the date of the insolvency event as a dummy variable.
The results are not as expected in particular presence in the lower tiers of English football is being shown as insignificant (with the highest tier being the base group) - I don't understand this because almost all insolvency events took place in the lower tiers so would expect this to return significant results.

Therefore I wanted to check with people that I have my data in my STATA data set laid out in the right way. I have a database that consists of the clubs that participated in the football league in every season from the 1996/97 season to the 2014/15, for every season a club has competed in the football league (the top 4 divisions) there is a seperate entry. It also includes which league they were in in that season. I also have column named insolvency event and if the club had an insolvency event in that season there is a 1 in the column and a 0 if not

Any help for me on this matter would be much appreciated,

Thankyou

[Clyde Schechter]

It's hard to give concrete advice based on a "view from the clouds" description of what you have done. The devil is usually in the details. If you want to show an example of your data, using the -dataex- command, and the commands you've used (between code delimiters, for maximum readability),* you could get commentary on that.

Finally, even when the failures occur predominantly in a given tier, it may well be that after adjusting for the various shocks you have included in your model, that the effect of tier is subsumed by those shocks and doesn't show up. So it may be that your expectations are wrong and your model is OK. But, as noted, I really can't say based only on what you've divulged.

Added: *Read FAQ #12 for information on how to get and use -dataex- and use code delimiters. Actually, invest the time to read the entire FAQ for good advice on how to post so as to maximize your chances of getting a helpful and timely response.

[Philip Murphy]

Hi Clyde,

Thanks very much for your reply I have run the following regression initally

xtprobit Insolvencyeventvalues i.LeagueNumber l.ProbitResidualsyearranking l2.ProbitResidualsyearranking l.Probitresidualsrelrev l2.Probitresidualsrelrev itvdummy

When I ran this regression, of all variables only the first lag of ProbitResidualsyearranking and the second lag of Probitresidualsrelrev were signifcant, I have then run a regression with out the second lag of ProbitResidualsyearranking and the 1st lag of Probitresidualsrelrev as follows

xtprobit Insolvencyevent i.LeagueNumber l.ProbitResidualsyearranking l2.Probitresidualsrelrev itvdummy

This causes the divisional dummy for tier 2 to become significant and also the ITV dummy to become significant whilst the residual variables also remain significant. I am not sure why taking away the insignificant residual variables causes some of the dummy variables to become significant.

I have attempted to attach an example of my data below using the dataex command

Thank you very much for any help you can provide

* Example generated by -dataex-. To install: ssc install dataex
clear
input int(TeamNumbers SeasonStart) byte(LeagueNumber LeaguePos) str10 Promotionorrelegation double(Relativewagespend RelativeRevenue) byte YearRanking double(ProbitResidualsyearranking Probitresidualsrelrev) byte Insolvencyevent float itvdummy
1 2006 4 20 "ɴone"                        .                  . 88                   .                   . 0 0
1 2007 4 17 "ɴone"                        .                  . 85                   .                   . 0 0
1 2008 4 16 "ɴone"                        .                  . 84                   .                   . 0 0
1 2009 4 15 "ɴone"                        .                  . 83                   .                   . 0 0
1 2010 4  5 "ɴone"                        .                  . 73                   .                   . 0 0
1 2011 4 14 "ɴone"                        .                  . 82                   .                   . 0 0
1 2012 4 18 "ɴone"                        .                  . 86                   .                   . 0 0
1 2013 4 15 "ɴone"                        .                  . 83                   .                   . 0 0
1 2014 4 17 "ɴone"                        .                  . 85                   .                   . 0 0
2 1996 3 16 "ɴone"                        .                  . 60                   .                   . 1 0
2 1997 3  9 "ɴone"                        .                  . 53                   .                   . 0 0
2 1998 3  7 "ɴone"                        .                  . 51                   .                   . 0 0
2 1999 3 16 "ɴone"                        .                  . 60                   .                   . 0 0
2 2000 3  7 "ɴone"                        .                  . 51                   .                   . 0 0
2 2001 3 21 "Relegation"                   .                  . 65                   .                   . 0 0
2 2002 4  4 "Promotion"                    .                  . 72                   .                   . 0 1
2 2003 3  9 "ɴone"                        .                  . 53                   .                   . 0 1
2 2004 3  8 "ɴone"                        .                  . 52                   .                   . 0 0
2 2005 3 17 "ɴone"                        .                  . 61                   .                   . 0 0
2 2006 3 19 "ɴone"                        .                  . 63                   .                   . 0 0
2 2007 3 21 "Relegation"                   .                  . 65                   .                   . 1 0
2 2008 4 21 "ɴone"                        .                  . 89                   .                   . 0 0
2 2009 4  2 "Promotion"                    .                  . 70                   .                   . 0 0
2 2010 3  6 "ɴone"                        .                  . 50                   .                   . 0 0
2 2011 3 11 "ɴone"       .13745736404009418 .10262551530158269 55                   .  -.5086503028869629 0 0
2 2012 3  2 "Promotion"   .36178182997759195 .12274970790487512 46 -.12458258867263794 -.41487255692481995 0 0
2 2013 2 10 "ɴone"        .4671034770503484 .19015394698082191 30 -.09649283438920975  -.7511547803878784 0 0
2 2014 2  1 "Promotion"    .7353217082634866  .2225843560383032 21  .33870047330856323  -.6444125771522522 0 0
3 2011 4 16 "ɴone"       .05335419020851822 .07473136040936805 84                   .                   . 0 0
3 2012 4 20 "ɴone"      .052898944554319545 .08202687001208653 88  -.8473091721534729  -.1686929613351822 0 0
3 2013 4 20 "ɴone"       .06350847797447153  .0653396196080636 88  -.9924400448799133 .028022225946187973 0 0
3 2014 4 15 "ɴone"       .05659415379218311 .06861541830021263 83 -.09304133802652359  .03129802644252777 0 0
4 2008 4 15 "ɴone"                        .                  . 83                   .                   . 0 0
4 2009 4  6 "ɴone"                        .                  . 74                   .                   . 0 0
4 2010 4 14 "ɴone"                        . .06811207528657698 82                   . -.47663983702659607 0 0
4 2011 4 11 "ɴone"                        . .07809020696802682 79                   .  -.2482493817806244 0 0
4 2012 4 24 "Relegation"                   .                  . 92                   .                   . 1 0
5 1996 1  3 "ɴone"        3.086516455147921 3.3675279653448786  3                   .                   . 0 0
5 1997 1  1 "ɴone"       3.2184053094697647  3.904296280732827  1  2.2733898162841797  1.6119434833526611 0 0
5 1998 1  2 "ɴone"       3.3579911405833007  4.289679658462849  2  1.5510777235031128  1.6203217506408691 0 0
5 1999 1  2 "ɴone"        3.576832493328338  4.785750093249399  2  1.5176948308944702  2.3532674312591553 0 0
5 2000 1  2 "ɴone"       3.2417149369942115 3.9727434211368764  2    1.61069917678833  1.5402607917785645 0 0
5 2001 1  1 "ɴone"        4.765553372629261  4.766826395109629  1   1.997707486152649   2.334343910217285 0 0
5 2002 1  2 "ɴone"       4.2863241324962384  5.185010834716778  2  1.4944182634353638   2.515652894973755 0 1
5 2003 1  1 "ɴone"        4.670130729397815  5.117723702541437  1  2.1012675762176514   2.685241222381592 0 1
5 2004 1  2 "ɴone"        4.191252355909486  4.977355022919434  2  1.5046478509902954   2.307997226715088 0 0
5 2005 1  4 "ɴone"        4.828703599772487  5.352097980824564  4   .6499357223510742  2.9196155071258545 0 0
5 2006 1  4 "ɴone"        4.824714392573421  4.962177793034608  4   .6954324841499329  2.7703702449798584 0 0
5 2007 1  3 "ɴone"        4.385273154855342  5.598863030032331  3  1.0648818016052246   3.407055616378784 0 0
5 2008 1  4 "ɴone"       3.9647711392572513  5.748861467080619  4   .8105418682098389  3.4565086364746094 0 0
5 2009 1  3 "ɴone"        3.960481168416521  6.302615788961089  3  1.0782434940338135   4.110808372497559 0 0
5 2010 1  4 "ɴone"       3.8961322128420237  5.047389665556874  4   .7911988496780396  2.7550368309020996 0 0
5 2011 1  3 "ɴone"        4.197019776704982  5.743025625645935  3  1.0296673774719238   3.551218032836914 0 0
5 2012 1  4 "ɴone"        4.370477989730699  5.215993790463957  4   .7243480682373047  2.9236409664154053 0 0
5 2013 1  4 "ɴone"        4.491817167772579  5.714893762407841  4     .71686851978302  3.5230863094329834 0 0
5 2014 1  3 "ɴone"        4.573839367712207  5.798154446973318  3  1.0109796524047852   3.606346845626831 0 0
6 1996 1  5 "ɴone"        2.025646150840778 2.7377432352399635  5                   .                   . 0 0
6 1997 1  7 "ɴone"         1.80748119412959  3.070871940348127  7   .5545910596847534   .9579390287399292 0 0
6 1998 1  6 "ɴone"       2.0730586017278543 3.0768744785091484  6   .5824620127677918  1.0848767757415771 0 0
6 1999 1  6 "ɴone"       2.2556981152355227 2.7841539977735223  6    .563277006149292   .7364053130149841 0 0
6 2000 1  8 "ɴone"        2.234283688198628 2.4878876489319617  8  .28618305921554565   .4401389956474304 0 0
6 2001 1  8 "ɴone"       2.4119940723415745  2.458281235349135  8  .22637543082237244   .5151430368423462 0 0
6 2002 1 16 "ɴone"       2.3311931683804588 2.2704061837513936 16   -.516074001789093  .32726797461509705 0 1
6 2003 1  6 "ɴone"       2.2416566574109247 2.4846242551475988  6   .6051141619682312   .8079395294189453 0 1
6 2004 1 10 "ɴone"        2.164411703140058  2.232856818146165 10    .060822494328022  .18510815501213074 0 0
6 2005 1 16 "ɴone"        2.348058642196122 1.9843501218416861 16  -.5557576417922974  .12429574131965637 0 0
6 2006 1 11 "ɴone"       1.2075613494183568 1.3278358394402658 11  .40105292201042175  -.3488488793373108 0 0
6 2007 1  6 "ɴone"       2.4059508950274573  2.310366013980368  6  .34410595893859863   .4864553213119507 0 0
6 2008 1  6 "ɴone"        2.830826416477315  2.462920319498851  6  .44198477268218994  .41517165303230286 0 0
6 2009 1  6 "ɴone"       2.9770423029050823  2.500667974382612  6  .45617440342903137   .4529193043708801 0 0
6 2010 1  9 "ɴone"       2.6878676824864054   2.35038140416271  9  .10739772021770477  .30263274908065796 0 0
6 2011 1 16 "ɴone"       2.1267104382886464  1.976262216934224 16 -.41362857818603516  .07673174142837524 0 0
6 2012 1 15 "ɴone"       2.0624860673921472  1.982137320481059 15  -.3876730501651764   .3054526150226593 0 0
6 2013 1 15 "ɴone"       1.8984670508142232  2.202978400648593 15  -.3359125852584839   .5002396106719971 0 0
6 2014 1 17 "ɴone"        2.050663374783515 2.0013667181207118 17  -.5705379247665405  .29862791299819946 0 0
7 1996 4 15 "ɴone"                        .                  . 83                   .                   . 0 0
7 1997 4  7 "ɴone"                        .                  . 75                   .                   . 0 0
7 1998 4 16 "ɴone"                        .                  . 84                   .                   . 0 0
7 1999 4  6 "ɴone"                        .                  . 74                   .                   . 0 0
7 2000 4 24 "Relegation"                   .                  . 92                   .                   . 0 0
7 2005 4 18 "ɴone"                        .                  . 86                   .                   . 0 0
7 2006 4 14 "ɴone"                        .                  . 82                   .                   . 0 0
7 2007 4 12 "ɴone"                        .                  . 80                   .                   . 0 0
7 2008 4 17 "ɴone"                        .                  . 85                   .                   . 0 0
7 2009 4 21 "ɴone"                        .                  . 89                   .                   . 0 0
7 2010 4 22 "ɴone"                        .                  . 90                   .                   . 0 0
7 2011 4 22 "ɴone"                        .                  . 90                   .                   . 0 0
7 2012 4 23 "Relegation"                   .                  . 91                   .                   . 0 0
8 1996 2  2 "Promotion"                    .                  . 22                   .                   . 0 0
8 1997 1 19 "Relegation"                   .                  . 19                   .                   . 0 0
8 1998 2 13 "ɴone"                        .                  . 33                   .                   . 0 0
8 1999 2  4 "ɴone"                        .                  . 24                   .                   . 0 0
8 2000 2 16 "ɴone"                        .                  . 36                   .                   . 0 0
8 2001 2 23 "Relegation"                   .                  . 43                   .                   . 0 0
8 2002 3 19 "ɴone"                        .                  . 63                   .                   . 1 1
8 2003 3 12 "ɴone"                        .                  . 56                   .                   . 0 1
8 2004 3 13 "ɴone"                        .                  . 57                   .                   . 0 0
8 2005 3  5 "Promotion"                    .                  . 49                   .                   . 0 0
8 2006 2 20 "ɴone"        .2177419986398328                  . 40                   .                   . 0 0
8 2007 2 18 "ɴone"        .3111383000882928 .27421609261811347 38  -.3886895179748535 -.43784451484680176 0 0
end
label var TeamNumbers "Team Numbers" 
label var SeasonStart "Season Start" 
label var LeagueNumber "League Number" 
label var LeaguePos "League Pos" 
label var Promotionorrelegation "Promotion or relegation" 
label var Relativewagespend "Relative wage spend" 
label var RelativeRevenue "Relative Revenue" 
label var YearRanking "Year Ranking" 
label var ProbitResidualsyearranking "Probit Residuals year ranking" 
label var Probitresidualsrelrev "Probit residuals rel rev" 
label var Insolvencyevent "Insolvency event"

[Clyde Schechter]

Actually, it would have been helpful had you posted the results of the regressions you did as well so that we could talk concretely about the specifics. I can't replicate your activities with what you have shown because I can't tell how you -xtset- your data, and also because your first regression uses a variable Insolvencyeventvalues which does not appear in your data example.

But I think your question can be given a general answer. There are two complementary aspects to this. Both of them reflect a misplaced focus on statistical significance.

First, a change from statistically significant to not statistically significant in your other variables does not necessarily mean that a meaningful or important change has occurred. To assess that, again, you need to ignore the p-values and look at the model coefficients. Did they change much? Or was it a tiny change in the coefficients that happened to flip the p-value from < 0.05 to > 0.05. It is not uncommon for meaningless small changes to do that.


Alternatively, perhaps the coefficients of the main predictor variables did change in a meaningful way. Selecting what variables to include in a model should be based on scientific considerations and an understanding of the underlying relationships. It is inappropriate to include or exclude variables from the model based on whether they turn out to have a statistically significant coefficient in the model. Covariates should be included if it is necessary to adjust for their effects to reduce the missing variable bias that would occur in their absence, or to reduce an extraneous source of outcome variation. A variable can be necessary in this way without its contribution to the overall regression being "statistically significant." The need for the variable is based on whether it is associated with both the predictor(s) of interest and the outcome variable to a sufficiently large extent. The term "sufficiently large" does not mean "statistically significant." Rather it means large enough that the variable could appreciably alter the outcomes of interest. This decision should be made before the analysis is run, and you should stick with it.

Whenever you estimates two models, one of which contains a covariate and the other excludes it, you are estimating two different things. The former estimates the association of the main predictor(s) with the outcome conditional on the included variable, the latter estimates the association with the marginal outcome. There is no reason to expect that these will be the same, or even close to each other. They can even have opposite signs. This phenomenon is sometimes known as Simpson's paradox (although it was initially described by other people). Wikipedia has a good page on Simpson's paradox, and I recommend you read it. The examples given there are analyses of simple contingency tables with dichotomous variables, but the same principles apply to all analyses of the associations among variables.


The bottom line lessons here are:

1. Selecting what variables to include in the model should be done before you actually run the model based on prior understanding of the underlying science. If there is good reason to include a variable, you should not change the decision based on a lack of "statistical significance" when the model is run.

2. Simpson's paradox arises commonly. The decision needs to be made in advance whether it is the conditional or marginal association that is of interest for your research goals. (Sometimes both are.)

3. p-values are overrated. Don't even look at them until you have understood the regression coefficients and what they mean, and how well your model predictions fit the data. Once you have done that, if you have nothing better to do, look at the p-values.

[Philip Murphy]

Thanks for your advice Clyde,

I have uploaded the correct example of my data , would you be able to confirm that it is set out in the right way please?
I am trying to predict the probability of an insolvency event. If an insolvency event has occurred for a particular team in a particular season start the insolvency event values variable takes a value of 1 and a 0 if not. I just wondered if i have it laid out correctly in this sense for the xtprobit command to work properly?
My data is declared as

xtset TeamNumbers SeasonStart

* Example generated by -dataex-. To install: ssc install dataex
clear
input int(TeamNumbers SeasonStart) byte(LeagueNumber Insolvencyeventvalues) float itvdummy double(Probitresidualsrelrev ProbitResidualsyearranking)
1 2006 4 0 0                   .                   .
1 2007 4 0 0                   .                   .
1 2008 4 0 0                   .                   .
1 2009 4 0 0                   .                   .
1 2010 4 0 0                   .                   .
1 2011 4 0 0                   .                   .
1 2012 4 0 0                   .                   .
1 2013 4 0 0                   .                   .
1 2014 4 0 0                   .                   .
2 1996 3 1 0                   .                   .
2 1997 3 0 0                   .                   .
2 1998 3 0 0                   .                   .
2 1999 3 0 0                   .                   .
2 2000 3 0 0                   .                   .
2 2001 3 0 0                   .                   .
2 2002 4 0 1                   .                   .
2 2003 3 0 1                   .                   .
2 2004 3 0 0                   .                   .
2 2005 3 0 0                   .                   .
2 2006 3 0 0                   .                   .
2 2007 3 1 0                   .                   .
2 2008 4 0 0                   .                   .
2 2009 4 0 0                   .                   .
2 2010 3 0 0                   .                   .
2 2011 3 0 0  -.5086503028869629                   .
2 2012 3 0 0 -.41487255692481995 -.12458258867263794
2 2013 2 0 0  -.7511547803878784 -.09649283438920975
2 2014 2 0 0  -.6444125771522522  .33870047330856323
3 2011 4 0 0                   .                   .
3 2012 4 0 0  -.1686929613351822  -.8473091721534729
3 2013 4 0 0 .028022225946187973  -.9924400448799133
3 2014 4 0 0  .03129802644252777 -.09304133802652359
4 2008 4 0 0                   .                   .
4 2009 4 0 0                   .                   .
4 2010 4 0 0 -.47663983702659607                   .
4 2011 4 0 0  -.2482493817806244                   .
4 2012 4 1 0                   .                   .
5 1996 1 0 0                   .                   .
5 1997 1 0 0  1.6119434833526611  2.2733898162841797
5 1998 1 0 0  1.6203217506408691  1.5510777235031128
5 1999 1 0 0  2.3532674312591553  1.5176948308944702
5 2000 1 0 0  1.5402607917785645    1.61069917678833
5 2001 1 0 0   2.334343910217285   1.997707486152649
5 2002 1 0 1   2.515652894973755  1.4944182634353638
5 2003 1 0 1   2.685241222381592  2.1012675762176514
5 2004 1 0 0   2.307997226715088  1.5046478509902954
5 2005 1 0 0  2.9196155071258545   .6499357223510742
5 2006 1 0 0  2.7703702449798584   .6954324841499329
5 2007 1 0 0   3.407055616378784  1.0648818016052246
5 2008 1 0 0  3.4565086364746094   .8105418682098389
5 2009 1 0 0   4.110808372497559  1.0782434940338135
5 2010 1 0 0  2.7550368309020996   .7911988496780396
5 2011 1 0 0   3.551218032836914  1.0296673774719238
5 2012 1 0 0  2.9236409664154053   .7243480682373047
5 2013 1 0 0  3.5230863094329834     .71686851978302
5 2014 1 0 0   3.606346845626831  1.0109796524047852
6 1996 1 0 0                   .                   .
6 1997 1 0 0   .9579390287399292   .5545910596847534
6 1998 1 0 0  1.0848767757415771   .5824620127677918
6 1999 1 0 0   .7364053130149841    .563277006149292
6 2000 1 0 0   .4401389956474304  .28618305921554565
6 2001 1 0 0   .5151430368423462  .22637543082237244
6 2002 1 0 1  .32726797461509705   -.516074001789093
6 2003 1 0 1   .8079395294189453   .6051141619682312
6 2004 1 0 0  .18510815501213074    .060822494328022
6 2005 1 0 0  .12429574131965637  -.5557576417922974
6 2006 1 0 0  -.3488488793373108  .40105292201042175
6 2007 1 0 0   .4864553213119507  .34410595893859863
6 2008 1 0 0  .41517165303230286  .44198477268218994
6 2009 1 0 0   .4529193043708801  .45617440342903137
6 2010 1 0 0  .30263274908065796  .10739772021770477
6 2011 1 0 0  .07673174142837524 -.41362857818603516
6 2012 1 0 0   .3054526150226593  -.3876730501651764
6 2013 1 0 0   .5002396106719971  -.3359125852584839
6 2014 1 0 0  .29862791299819946  -.5705379247665405
7 1996 4 0 0                   .                   .
7 1997 4 0 0                   .                   .
7 1998 4 0 0                   .                   .
7 1999 4 0 0                   .                   .
7 2000 4 0 0                   .                   .
7 2005 4 0 0                   .                   .
7 2006 4 0 0                   .                   .
7 2007 4 0 0                   .                   .
7 2008 4 0 0                   .                   .
7 2009 4 0 0                   .                   .
7 2010 4 0 0                   .                   .
7 2011 4 0 0                   .                   .
7 2012 4 0 0                   .                   .
8 1996 2 0 0                   .                   .
8 1997 1 0 0                   .                   .
8 1998 2 0 0                   .                   .
8 1999 2 0 0                   .                   .
8 2000 2 0 0                   .                   .
8 2001 2 0 0                   .                   .
8 2002 3 1 1                   .                   .
8 2003 3 0 1                   .                   .
8 2004 3 0 0                   .                   .
8 2005 3 0 0                   .                   .
8 2006 2 0 0                   .                   .
8 2007 2 0 0 -.43784451484680176  -.3886895179748535
end
label var TeamNumbers "Team Numbers" 
label var SeasonStart "Season Start" 
label var LeagueNumber "League Number" 
label var Insolvencyeventvalues "Insolvency event values" 
label var Probitresidualsrelrev "Probit residuals rel rev" 
label var ProbitResidualsyearranking "Probit Residuals year ranking"

Thanks

[Clyde Schechter]

Well, the overall layout is fine. There is a problem with the example you show, which may or may not also occur in the full data set. In the example data, whenever Insolvencyevent == 1, there are missing values for Probitresidualsrelrev and ProbitResidualsyearrank. In any regression model, all observations that have missing values for any model variable are omitted from the estimation sample. The result here is that in the estimation sample there are no observations with Insolvencyevent == 1. Since Insolvencyevent always is zero in the estimation sample, there is no variation in the outcome, and the regression cannot proceed. I suspect this is just a coincidence in the example you showed and that in your full data set you won't have this problem. But if you do get an error message saying that your outcome does not vary, then this is probably what's going wrong.