what's wrong with the missing p-values

[Tracy Chen]

Dear all,

I have a question related to the missing p-values. my regression function is xtreg y_it (performance measures) CDO_ijt controls_it i.year, fe. However, the problem came when I used the loop command (displayed as follows) to obtain the coefficients of each CDO fixed effect, the p-value of some coefficients was missing. There will be more missing value if I cluster by company.

encode Company_id,gen(company)
gen lnassets = log(Totalassets)

xtset company Year

tempname memhold
postfile `memhold' cdo_id CDO_b CDO_pvalue using "result2_post.dta", replace

levelsof CDO_id, local(levels)

foreach x of local levels {
quietly xtreg ROA CDO CF Leverage Investment Cashholdings lnassets i.Year if CDO_id == `x', fe
matrix CDO_coef = e(b)
matrix CDO_se = e(V)

scalar CDO_b = CDO_coef[1, "CDO"]
scalar CDO_se = sqrt(CDO_se[1, 1])

scalar CDO_pvalue = 2 * (ttail(e(df_r), abs(CDO_b / CDO_se)))

post `memhold' (`x') (CDO_b) (CDO_pvalue)
}
postclose `memhold'

use result2_post.dta, clear
list

any suggestions on this issue will be highly appreciated!

[Clyde Schechter]

I don't see anything obviously wrong with your code. I suspect that the problem lies with the data in some way, or that the data is valid but not suitable for use with this code. As you don't provide any example data, it's impossible to troubleshoot this aspect of things.

Here's something you can do on your own first. Get rid of the -quietly- in your -xtreg- command. Look at the -xtreg- output for the values of CDO_id where your p-value is missing. Look at it carefully: don't just look at the coefficient of CDO and p-value. Are there other missing statistics? If so, which ones? Do the sample sizes (both observations and groups) in the -xtreg- output correspond to what you expect them to be? Did Stata issue any messages about variables being omitted due to colinearity, or for some other reason? You may be able to figure out what's wrong just from these things.

If after doing that you are not sure what is happening, when posting back, show some of the output from the -xtreg-'s where things have gone wrong. And, crucially, also show example data. Be sure your example data includes some of the CDO_id's that produce problematic outputs. And use the -dataex- command to show the example data so that it will contain all of the needed metadata and will be importable into Stata for those who want to help you to work with. If you are running version 18, 17, 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

[Tracy Chen]

Hi Clyde, thanks for your reply and the suggestions provided. I figured out that the collinearity caused the problem, as I noticed that Stata omitted some company-year observations because of collinearity. But what confuses me is that if I run the following running regression:

reg ROA CDO CF Leverage Investment Cashholdings lnassets i.company i.Year, cluster (company)

I suspect the above regression is equivalent to "xtreg ROA CDO... i.Year, cluster(company")? Then conduct the VIF test, the VIF of 'lnassets' is greater than 10. However, after I omitted this variable, and then ran the commands demonstrated in my first post, the problem remained unsolved.

I post my sample data which includes some CDO_ids that produce problematic outputs below, i.e., CDO_id 1978032 if clustering by company, or CDO_id 2162824 and 2264218 if not using cluster command. I would appreciate any advice on how to correct collinearity in this case.

* Example generated by -dataex-. For more info, type help dataex
clear
input long company int Year long CDO_id byte CDO double(ROA CF Leverage Investment Cashholdings) float lnassets
 3 2012 2099971 0   -.02180253406645948   .019565658615790543     .819335705812574  .08556931195453622  .10525674852851634  10.64137
 3 2013 2099971 0   .002821014080324664   .056068811723478815    .8298892119116839  .07916533928002549  .09198789423383243 10.606857
 3 2014 2099971 0   .019735153723759176    .08007633891296569    .8304290874449252   .0801558586139716  .08938014393065882 10.570445
 3 2015 2099971 0   .008390457910611462    .06584324766142947    .8647743376132606  .10480428662246843   .0783761692852602 10.504246
 3 2016 2099971 0   -.03128114273059462  .0022532451628704386    .8744156778137361  .11486651971589518  .08689688954200343 10.494713
 3 2017 2099971 0   -.03506281710558106 .00039416633819471815    .8901417238613067  .10726251478123769   .0811489948758376  10.40765
 3 2018 2099971 0    .03699148242673964    .10310338381005796    .8574792305300102   .0991306786315199  .08641802206019816 10.389642
 3 2019 2099971 1   .009004992867332383    .05906581368854614     .872597380506889  .10538077293839278  .06261502059416353  10.42371
 3 2020 2099971 1  .0013293645059676907    .04822302064845678    .8850082947699293   .0822475217523051   .0616423531448855 10.451695
 3 2021 2099971 0  -.012407851227133452    .03205509314308363    .9065814393939394  .10483551327784384 .058214427269124056  10.40314
 3 2022 2099971 0   -.01423246357167062   .021671297285744816    .9377142411966345   .1945287454584313  .08993374652703569  10.55485
 3 2023 2099971 0    .00555815978035224    .04538862153075351    .9435356922866334   .2545981936844881   .0600237326125651  10.70994
14 2012 2099971 0    .06385298415196133     .5160075329566854    .6960208062418726  .30836522689994533  .33351558228540185 11.396054
14 2013 2099971 0     .0446310780248775     .4026939970717423    .6446269701678592   .2603806734992679   .3561346998535871 11.349182
14 2014 2099971 1    .02895572796731262      .338722326114496    .7011297317556353   .2038366411292755   .4428424926102431 11.346646
14 2015 2099971 0    .03206453754052743     .3454319453076445    .7196433304305024  .20267246737103792  .40149160969546305  11.26884
14 2016 2099971 0 -.0008968729920753909     .1936431275290432    .7368389423076923  .19109776791541574   .4678240438585041  11.22129
14 2017 2099971 0   .009797042696395624     .2565171317555634    .7180236747297993  .06344048039561992   .5836100317908867 11.251067
14 2018 2099971 0    .07829675578596085     .6566229556505083    .7225055344718533  .21482245469279504     .57882716958892 11.270968
14 2019 2099971 0    .07766433405988299     .6408929627439385    .7241248888972939  .19729450029568302   .6123595505617978 11.270255
14 2020 2099971 0     .0382769010775752    .41756382651491847    .7113567044470385  .16264226391879422   .7191633343586589  11.26861
14 2021 2099971 0    .07837619122389573     .6953204876130554    .6998270053719384  .19441604404246954   .7278018088871412   11.3241
14 2022 2099971 0    .08182517115555936     .7087039390088945    .7030557062929211  .20640088945362134   .5562261753494282  11.31378
14 2023 2099971 0    .11814669166399927     .9428074947113931    .6635833362095744  .23360531882744032   .5271985494106981  11.37912
 5 2012 1950347 0   .018165061009712716     2.723963599595551   .22739047189998873  .28816986855409504   .8149646107178968 11.751525
 5 2013 1950347 0   .018458549222797927             2.5859375    .2307477626658232         .2021484375         .6591796875 11.724158
 5 2014 1950347 0   .026272366080070796     3.119301648884578   .20047583759116971   .2793404461687682   .6372453928225025 11.594312
 5 2015 1950347 0   .020744152270295062           2.482421875   .21949965729952023         .2958984375         .4833984375 11.558434
 5 2016 1950347 0    .01728201823036553      2.12112676056338   .25271749950228944  .29389671361502345  .40938967136150234  11.59552
 5 2017 1950347 0    .02836633399156749     3.425373134328358    .2343519338647771   .2789179104477612   .5755597014925373 11.630015
 5 2018 1950347 0   .019242933121898637     2.555980861244019   .25034927041291527  .26507177033492824  .47751196172248805 11.628475
 5 2019 1950347 0    .04040850354314297    3.3746928746928746   .23354475438482947  .26597051597051596   .2076167076167076  11.69483
 5 2020 1950347 0     .0442585346107138     4.318620689655172   .22820849759088918  .21241379310344827                 .26 11.743934
 5 2021 1950347 0   .016080048270313757     2.142857142857143   .26716252571609433   .2753391859537111   .6089385474860335  11.50731
 5 2022 1950347 0   -.01315453775423772     -.361998361998362    .3494741270509045    .343980343980344   .6027846027846028  11.49261
 5 2023 1950347 1  -.001818850254445541     .5048923679060665   .34300162995778827  .26125244618395305   .7064579256360078 11.545993
40 2012 1950347 0    .01193274734000043     .9465064162592594    .2746583504475039  .11087502664023094  1.5405157548711257  8.650051
40 2013 1950347 0   .009928430654033885      .743973215362572   .24436980207378886  .07713608930785448  1.7030533135857742  9.375039
40 2014 1950347 0   .010086172690015284     .7950652701201807    .4087537292768794 .061704045701535495  1.5831079634029117  9.413333
40 2015 1950347 0   .009152058665044652      .782465142903872    .4989083905274559  .06520116939935397  1.4208662141718116  9.447694
40 2016 1950347 0   .008119255264229854     .7839686031991437    .5162744995717069  .05518225605042517  1.7742111607842714  9.499447
40 2017 1950347 0   .007655483389456393     .8035117849435984   .48894071267313727  .07370956679807515  1.8756823669140565  9.532275
40 2018 1950347 0   .011258378830779606    1.0551637674345713    .1077623881166026  .08196207490988873  1.9611290214252133  9.494501
40 2019 1950347 0   .011146938144420786     .8592905015544885   .20780431306551264  .07841229471473891  1.6242527367596946  9.510287
40 2020 1950347 0   .009668109406567487      .894177520994409   .24000384119286497   .1002624578668526   8.671119420549873  9.714252
40 2021 1950347 1   .008375088004775731      .804080800176485   .24035817629529388  .11388279562283815   9.435410223644832  9.775406
40 2022 1950347 0    .00399029101531472    .44947439537371414    .5360709969915317  .10706731117241601    2.97100126273797  9.798986
40 2023 1950347 0    .00883919075915749     .7434821319718191   .40172780501010896  .14807945943349873  3.6040586820551286  9.837464
 6 2012 2264218 0   .006558923371374358    .25860323886639675     .863571278163824   .5381723539618276   .5431607865818392 12.113667
 6 2013 2264218 0  .0023880873471061807    .18001674574379012    .8576014423446896   .5133128663131454  .30873569634384596  11.92614
 6 2014 2264218 0  .0075842011198237825     .2070433240435774    .8385130043614276   .5008360780339498   .2825437040790474 11.929205
 6 2015 2264218 0   .008128338199405982    .24778868290318673    .8536599370679161   .2838361807827457   .3865261117169514  11.97402
 6 2016 2264218 0   .006516906088146194    .29115313185885533    .8337826705609226  .27638021273003544   .5009285834880972  12.00596
 6 2017 2162824 0  .0055579486443152176     .3031423290203327    .8048110164465595  .44057844949440034  .46232467108839836 12.026635
 6 2018 2162824 0   .007061033493785955    .32424006235385816    .8032854940102004   .4129829640351854   .5051775971495379  12.09441
 6 2019 2162824 1   .009493811031642346      .337879727216367    .7338748384714787  .41568505889646623  .36732796032238063 12.104283
 6 2020 2162824 1   .005956138665495567     .2503087299325544    .6233090797294527   .4103733257338273  1.4838985465944714 12.112668
 6 2021 2264218 1    .01680266206881404    .35435460062325735    .5388036329470717  .41987862883385274   .4151221912415942  12.11239
 6 2022 2264218 1   .008935180840970463    .25703659876595386    .6583206304933026    .298537739836024   .4708815822838306 12.164344
 6 2023 2264218 1   .005193694244164731    .21502418666413733    .6470914810930849  .26169022099971545   .6587309115052642 12.187868
29 2012 2162824 0   .046992700729927006      .884808189541449    .2629084720935743   .7488259194765448  2.3309060207904597  5.836272
29 2013 2162824 0     .0652066317342659    1.1791420354689248    .1916200248756219   .3085956223038824  1.4876178303243328   6.06461
29 2014 2162824 0   .014923178601397562    .34601320899118454   .34307363179096045   .4066441792567816  1.1310411292837097  6.512213
29 2015 2162824 0    .01959392830474757    .38401991923770606   .12454461660009655    .486042168770849  1.8013024116961678   6.95904
29 2016 2162824 0    .00784669630328481     .2788852631416155   .11106545933224302  .16840135475789114  3.7929818592350877  7.470373
29 2017 2162824 0    .03643282481545956     .6195984115442698  .057312129312595335   .6943285418647576  1.6682756306281115  7.922433
29 2018 2162824 0   .014016813746765042    .25839161371912667 .0029433332592618032   .4248411573798967  1.2344509454373696  8.208069
29 2019 2162824 0    .00374706929065406    .13615598766029258   .00492132055123276  .13327528941343395   .8192326020516018   8.47904
29 2020 2162824 0   .007563606228063122     .3133102168806674    .7312205856111069  .08095515433896332  1.2528397366421489  8.971106
29 2021 2162824 0    .02033203573724038     .7841970723908808    .3096225544427904 .012831187863245016   .8680410997685224  9.013521
29 2022 2162824 1   .017879157400265314     .7481750161418967   .09563065765382016  .16617038246800103  1.5976147973717194  9.195785
29 2023 2162824 1   .006556226307893865    .36904231021285017  .028555746879035726   .1816302868377275  2.2599183344255684  9.330026
 7 2012 2264218 1   .029267337077184277    1.5056396148555709    .7673364296009757   .2896836313617607   6.121045392022008 11.939108
 7 2013 2264218 1   .034940505297473515    1.6461935483870969    .7558330306711586  .25961290322580644  5.0286451612903225  11.94064
 7 2014 2264218 1     .0369886174365034    1.7513966480446928    .7482218541433234   .3034535297105129   5.659725749111224 11.977307
 7 2015 2264218 1     .0320317153067302    1.5107108081791627    .7189106137655344  .32643622200584227   5.540895813047712 11.990302
 7 2016 2264218 0    .03382779606401792    1.4595082337017822    .7194410991898401  .31017369727047145   5.686442589668396 11.975986
 7 2017 2264218 0   .015165897701936024       .93993993993994    .7638958923238044  .24532224532224534   7.606144606144606 12.107334
 7 2018 2264218 0   .036696323474830594     1.860054347826087    .7340508035746245   .2966485507246377   6.214900362318841 12.147394
 7 2019 2264218 0    .03408111092622566    1.6439801406702523    .7358726015478316   .3402978899462143  5.0570955730244105 12.197642
 7 2020 2264218 0   .016382134850836352     .9328015952143569    .6610729347922258    .294715852442672   6.573280159521436 12.161948
 7 2021 2264218 0    .04274773532469186    1.9556936647955092    .6485141453363975   .3107457898957498   4.416198877305534 12.147108
 7 2022 2264218 0    .03290505092969687    1.7526366251198466    .6399492947896025  .35570469798657717   6.503163950143816 12.338653
 7 2023 2264218 0    .03207102042066884    1.9511483067341378    .6366426647326979   .3042039704165045   9.068898404048268  12.47269
 2 2012 1978032 0    .11587113749382379    1.5195031814833078   .16474936519404518  .26389440709809187   7.787252903465391  8.634149
 2 2013 1978032 0    .10553327848944255    1.5425968817583058    .1554293377706365  .24219012891329886   9.215228732060666  8.761198
 2 2014 1978032 0     .0917417550938891    1.2391753504856835   .15500580213494414  .28583983213960945   4.605344926753874  8.833418
 2 2015 1978032 0    .09872503409584975    1.4269581282700161   .14638238541693657   .2390274953035972   4.702501125584139  8.862027
 2 2016 1978032 0     .1081096543934854     1.684368888693257    .2511203942709228  .20027322060756217   6.375870124316949  8.983475
 2 2017 1978032 0    .03439992840551766    1.1842438932759205    .4358656462267796   .1843197532023726   .9462965906381625  9.958983
 2 2018 1978032 0    .07373290017411371     1.997736345302202    .3597937076550918   .2208003271167294    .707416782751523  9.925169
 2 2019 1978032 0    .06371401268314651    1.7821980362118017   .31927706745850715  .22571678924973915   .5314162668679799  9.970802
 2 2020 1978032 0    .05686261933298594    1.8130954048909431   .31328328447842546   .1478652210812263   .9422601714676845  9.974347
 2 2021 1978032 0   .026574292137633466     1.245574772959363   .15779451805028843    .173656969931548   .9994507468478578 10.865173
 2 2022 1978032 1    .05464080703983015     2.101383248435017   .15987916559103815   .2912201037436326   .6124055929611577 10.825807
 2 2023 1978032 1    .06792938741961745    1.6038550762689494   .17173030802257896   .3607914707138275   .2740153593276698 10.795372
39 2012 1978032 0    .05759867398736144     .5356364768129473     .317668207498716  .16588857765328355  .49455337690631807  9.868171
39 2013 1978032 0    .06911868262824332     .5947567909033481    .2645251886958048  .19425142135186355  .44314592545799114  9.823416
39 2014 1978032 0    .08837832473203652     .7986301369863014   .29988340458608626  .21746575342465754   .8414383561643836  9.911058
39 2015 1978032 0    .11753849142746127    1.0397260273972602   .28836587719949514   .2054794520547945   1.140068493150685  9.932512
end
label values company company
label def company 2 "ADI", modify
label def company 3 "AES", modify
label def company 5 "ALL", modify
label def company 6 "ALLY", modify
label def company 7 "APX", modify
label def company 14 "CAT", modify
label def company 29 "LOB", modify
label def company 39 "TEL", modify
label def company 40 "TRMK", modify

[Clyde Schechter]

I suspect the above regression is equivalent to "xtreg ROA CDO... i.Year, cluster(company")? Then conduct the VIF test, the VIF of 'lnassets' is greater than 10. However, after I omitted this variable, and then ran the commands demonstrated in my first post, the problem remained unsolved.

I was not referring to "multicolinearity," which is what VIF purports to measure. I was referring to true colinearity, which is what Stata gives you warning messages about in the regression output.

So let's look at one of the CDO_id's where you get a problem:

2162824
note: 2021.Year omitted because of collinearity.
note: 2022.Year omitted because of collinearity.
note: 2023.Year omitted because of collinearity.

Fixed-effects (within) regression               Number of obs     =         16
Group variable: company                         Number of groups  =          2

R-squared:                                      Obs per group:
     Within  = 1.0000                                         min =          4
     Between = 1.0000                                         avg =        8.0
     Overall = 0.2368                                         max =         12

                                                F(14, 0)          =          .
corr(u_i, Xb) = -0.6220                         Prob > F          =          .

------------------------------------------------------------------------------
         ROA | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
         CDO |  -.0118037          .        .       .            .           .
          CF |   .0235704          .        .       .            .           .
    Leverage |  -.0053773          .        .       .            .           .
  Investment |   .0863736          .        .       .            .           .
Cashholdings |  -.0079448          .        .       .            .           .
    lnassets |   .0087844          .        .       .            .           .
             |
        Year |
       2013  |   .0402117          .        .       .            .           .
       2014  |  -.0048538          .        .       .            .           .
       2015  |  -.0077119          .        .       .            .           .
       2016  |    .021714          .        .       .            .           .
       2017  |  -.0242974          .        .       .            .           .
       2018  |  -.0211712          .        .       .            .           .
       2019  |  -.0090447          .        .       .            .           .
       2020  |  -.0018568          .        .       .            .           .
       2021  |          0  (omitted)
       2022  |          0  (omitted)
       2023  |          0  (omitted)
             |
       _cons |  -.0806557          .        .       .            .           .
-------------+----------------------------------------------------------------
     sigma_u |  .03048564
     sigma_e |          .
         rho |          .   (fraction of variance due to u_i)
------------------------------------------------------------------------------
F test that all u_i=0: F(1, 0) = .                           Prob > F =      .

Notice that it is a few of the Year indicators that are being omitted because of collinearity. Now, there are two ways a year indicator can be caught in a collinearity relationship. One of them is when one of the explanatory variables in the model is the same for all companies in that year. This is not, as far as I can see, the case in your data. The other way is when the model has too many variables relative to the sample size. And this, is, indeed the root of your problem. Look at the Number of obs = 16. You're trying to model 17 variables plus a constant, so 18 variables in all, with 16 observations. By linear algebra, the model is unidentifiable, and there have to be some collinearities among all of these variables because they are more numerous than the observations. So Stata throws away three variables to solve that problem, and it chose the last three year indicators to do that. So why doesn't Stata give you real results with this. Well, the problem is that while you have now brought things down to 15 variables with 16 observations, which is identifiable. Indeed, it is. Notice that you do get coefficients for the non-omitted variables and the constant. But, remember that the denominator degrees of freedom for the t-statistics is equal to the number of observations minus - number of parameters - 1. The number of parameters is the number of variables (including the constant) plus the number of panels minus 1. With 16 observations, 15 variables and 2 groups you don't have any degrees of freedom for the denominator. Notice that the model F statistic says F(14, 0)--that zero means that have only zero degrees of freedom for the denominator of any F or t-statistic in this model. In other words, it isn't possible to calculate those statistics, and, consequently, you get no p-value either.

By the way, if you go to clustered errors, the problem will be even more severe because the df available with clustered errors is even smaller: you don't get any credit for observations, only for groups!

tl;dr For the groups where you are getting no p-values, the problem is that you don't have enough observations in those groups to calculate them. You need more data in these smaller groups, or a model with fewer variables.

[Tracy Chen]

Thank you very much, Clyde!