Generating index numbers with panel data

[Sebastian Kruk]

Hello, Would anyone know whether there exists a possibility to generate index numbers over time with panel data stored in long format. The dataset looks like this:

* Example generated by -dataex-. For more info, type help dataex
clear
input str21 Country float year double(ci co2pc res_share ei gdppc pob_tot gdp) float co2 long pais float(ln_co2pc co2pc_gr ln_co2pc_gr ln_gdppc gdppc_gr ln_gdppc_gr)
"Argentina" 1970                  . 3286.2147819625325  7.709124606047121                  .                  . 23880.561                  .  78476656 1  8.097491            .            .         .            .           .
"Argentina" 1971                  . 3431.2368952227775 7.0252528343576035                  .                  . 24259.561                  .  83240304 1 8.1406765    .04413044    .04318523         .            .           .
"Argentina" 1972                  .  3419.874882656063  6.440856539585154                  .                  . 24653.175                  .  84310776 1   8.13736  -.003311346  -.003316879         .            .           .
"Argentina" 1973                  . 3543.3444355322176  6.250197582794806                  .                  . 25056.478                  .  88783728 1  8.172827   .036103528   .035467148         .            .           .
"Argentina" 1974                  . 3482.0688573090265    7.4318158578253                  .                  . 25462.302                  .  88661488 1  8.155382  -.017293148   -.01744461         .            .           .
"Argentina" 1975                  .  3340.517628680062 6.2682545487921555                  .                  . 25865.776                  .  86405080 1  8.113881   -.04065147   -.04150105         .            .           .
"Argentina" 1976                  .  3433.875534164317  5.818029016156285                  .                  . 26264.681                  .  90189648 1  8.141445   .027947137    .02756405         .            .           .
"Argentina" 1977                  .   3528.97699723825  6.269955537803029                  .                  . 26661.398                  .  94087464 1  8.168763    .02769508      .027318         .            .           .
"Argentina" 1978                  . 3462.9221273327285  6.993182276995134                  .                  . 27061.047                  .  93710296 1  8.149868   -.01871785   -.01889515         .            .           .
"Argentina" 1979                  . 3599.0624575541533  6.961591968522468                  .                  . 27471.036                  .  98869976 1  8.188429    .03931371    .03856087         .            .           .
"Argentina" 1980                  . 3478.8388622988623  8.650819515892865                  .                  . 27896.528                  .  97047528 1  8.154454   -.03340414  -.033974648         .            .           .
"Argentina" 1981                  . 3364.5341839169387   9.00704028733405                  .                  . 28338.515                  .  95345904 1  8.121045  -.032857135   -.03340912         .            .           .
"Argentina" 1982                  .  3265.577211153991  9.230390290558024                  .                  . 28794.548                  .  94030816 1  8.091192   -.02941179  -.029852867         .            .           .
"Argentina" 1983                  . 3300.7680005009056   9.78930423769241                  .                  . 29262.047                  .  96587232 1  8.101911   .010776284   .010718346         .            .           .
"Argentina" 1984                  .  3216.109174984265 10.091899747216608                  .                  . 29737.093                  .  95637736 1  8.075928   -.02564822  -.025982857         .            .           .
"Argentina" 1985                  .  2934.080594466689 10.376461253161487                  .                  . 30216.279                  .  88657000 1  7.984149   -.08769248   -.09177828         .            .           .
"Argentina" 1986                  .   3122.23060789614  9.801741388220577                  .                  . 30698.964                  .  95849248 1  8.046303    .06412572    .06215334         .            .           .
"Argentina" 1987                  . 3236.0833718548924  9.754464439775605                  .                  . 31184.415                  . 100915368 1  8.082119     .0364652   .035816193         .            .           .
"Argentina" 1988                  . 3301.3172619061534   7.66531704748684                  .                  . 31668.949                  . 104549248 1  8.102077   .020158285    .01995754         .            .           .
"Argentina" 1989                  .  3135.451134904495   6.80966188113748                  .                  . 32148.134                  . 100798904 1  8.050529   -.05024241     -.051548         .            .           .
"Argentina" 1990  204.6198461626269  2894.324455169788  9.409362366578138 .09709393640646029  14144.88628277752 32618.651  461387.1090926073  94408960 1  7.970507   -.07690334   -.08002138  9.557109            .           .
"Argentina" 1991 199.54222775906806 3037.4171077691212   8.64723890604451 .09333051273348351 15221.926415678638     33079 503526.10390423366 100474720 1  8.018763    .04943905    .04825544  9.630492    .07614343   .07338333
"Argentina" 1992 190.20665612421567   3083.14814975348  9.544988414512808 .08893535966003073 16209.465076448276 33529.326  543492.4388338492 103375880 1  8.033707   .015055898   .014944077  9.693351   .064876065   .06285858
"Argentina" 1993 181.80031741842453  3147.360211468547  10.53164462652347  .0831619939836647  17312.18215766205 33970.111  588096.7495479994 106916176 1  8.054319   .020826785   .020612717  9.759166    .06802921   .06581497
"Argentina" 1994 178.32791768373076   3226.34038081271 10.582949756812456  .0836220907582501 18092.177729202835 34402.672  622419.2561834699 110994728 1  8.079103     .0250941    .02478409  9.803235    .04505472   .04406929
"Argentina" 1995   183.547777713249 3186.8800273952697 10.687010551728543 .08744539467786108 17362.672907835655  34828.17  604710.1236884946 110993200 1  8.066797  -.012230685  -.012306213  9.762078   -.04032156  -.04115677
"Argentina" 1996 183.37542276802483  3319.985887377269  9.797564549518539 .08533674319897422 18104.857440885888 35246.374  638130.5765781469 117017464 1  8.107716    .04176683    .04091835  9.803936    .04274599   .04185772
"Argentina" 1997 172.97320227619076 3346.6356002165617 10.347988770369005 .08210570575154844  19347.71141527982 35657.429  689889.6461028297 119332424 1  8.115711   .008027056  .0079956055   9.87033    .06864754   .06639385
"Argentina" 1998 170.83766392139609 3393.9318747424154 10.492728929325343 .08050634623677688 19866.414681858405 36063.459  716451.6313561987 122396920 1  8.129745   .014132484   .014033318  9.896786    .02680954   .02645588
"Argentina" 1999  184.1320666446067  3495.072379540426  9.080366578363538  .0851884194256319 18981.334665005772 36467.218  692196.4691597224 127455568 1  8.159109    .02980039   .029364586  9.851212   -.04455157  -.04557419
"Argentina" 2000  185.1119141988014 3447.7929364362926  8.372645721130013 .09184412241756863 18625.451264760446 36870.787  686735.0463618629 127122840 1   8.14549  -.013527458  -.013619423  9.832284  -.018749123 -.018927574
"Argentina" 2001 187.64419915158848 3304.5843308712447 10.170165512804576 .09165211991840887  17610.90588364863 37275.652  656457.9991236388 123180536 1  8.103066   -.04153631   -.04242325  9.776274   -.05447092  -.05601025
"Argentina" 2002 199.52813419108566  3097.309604213842 10.616297441257245 .09951782014747247 15523.172292322377 37681.749  584940.2820030465 116712040 1  8.038289   -.06272339  -.064777374  9.650089   -.11854777  -.12618446
"Argentina" 2003 196.67830826460383  3287.441671363743  9.689462166476972 .09756597003793789 16714.815682372755 38087.868  636631.6933545434 125211648 1  8.097865     .0613862    .05957603  9.724051    .07676546   .07396126
"Argentina" 2004 196.61858647318198 3545.5774453987665  8.301036549493329 .09763284369925387 18032.768463028136 38491.972  694116.8187613621 136476272 1  8.173456    .07852178    .07559109  9.799946    .07884938    .0758953
"Argentina" 2005 187.96349226145168 3651.4928625726975  8.833553660790352 .09181892253460741 19426.606830083678 38892.931  755557.6790065732 142017264 1  8.202891   .029872544    .02943516  9.874399    .07729475  .074453354
"Argentina" 2006 183.38124030759045  3810.264909826442  8.940565296669032 .08846315899417256  20777.83367281942 39289.878  816358.5501093668 149704848 1  8.245454    .04348141    .04256248  9.941642    .06955548   .06724262
"Argentina" 2007  184.1718086758698  4129.927332036168 7.9013014347645445 .08806036518110864  22424.31869312076 39684.295  889893.2781918187 163893248 1  8.326015    .08389506    .08056164   10.0179    .07924238   .07625961
"Argentina" 2008 182.97659566051888  4227.428750391579   8.36635457791311 .08646843996249806  23103.65834018923  40080.16  925998.3228601188 169436016 1  8.349349    .02360851    .02333355 10.047747    .03029477   .02984524
"Argentina" 2009 181.71535448593897  3910.529161168259 10.131299840254844 .08564928413318539 21520.081075321865 40482.788  871192.8799150671 158309120 1  8.271428   -.07496273   -.07792091  9.976742   -.06854227  -.07100487
"Argentina" 2010 178.96412376453296 4198.4558331262715  10.52667545073768 .08259488230399488  23459.76246418121 40895.752  959404.6277140637 171699008 1  8.342472    .07362857    .07104397 10.063042    .09013356    .0862999
"Argentina" 2011 176.94535295847413  4355.093244030884 10.717426692686459 .08118955136200495 24612.645493170683   41320.5 1017006.8181005592 179954624 1  8.379102    .03730834   .036629677 10.111015      .049143   .04797363
"Argentina" 2012 181.66480708540726  4379.287095250712  10.18123911870099 .08283360296055381 24106.414255524178 41755.196 1006568.0520966062 182857984 1  8.384642     .0055553   .005539894 10.090233  -.020567933  -.02078247
"Argentina" 2013 182.46356185744366  4457.283246983502 10.188087989809782 .08305015736672737 24428.347236068523  42196.03 1030779.2728235644 188079664 1  8.402295   .017810239   .017653465   10.1035    .01335466  .013266563
"Argentina" 2014 184.22683520292824   4341.85328039038 11.106787466260853 .08518387971167299 23567.974098929575 42637.511 1004879.7548908248 185125824 1  8.376057  -.025896933   -.02623844 10.067644  -.035220277 -.035855293
"Argentina" 2015 183.68932027418938  4402.209581689255 10.037183620322597 .08393759775892923  23965.51729364649 43075.416 1032324.6270790168 189627008 1  8.389862   .013901046    .01380539 10.084372   .016867941  .016727448
"Argentina" 2016 186.71095042346815  4337.927774633249 10.560352900569773 .08586851188923822 23233.387033779487  43508.46 1010848.8904237134 188736560 1  8.375152  -.014602168  -.014710426 10.053346   -.03054932 -.031025887
"Argentina" 2017 179.24657937714292  4233.922757918741 11.661660055419597 .08260988244904152 23620.661396334795  43937.14 1037824.3066633574 186026464 1  8.350884   -.02397574  -.024267197 10.069877   .016668873   .01653099
"Argentina" 2018  176.6357165095495  4029.816551923169  11.49455536956859 .08332120526769808 22814.279193105907  44361.15 1012067.6614272501 178767296 1 8.3014765   -.04820735   -.04940796 10.035142   -.03413885 -.034734726
"Argentina" 2019  175.2078699541537 3874.1447456663263 11.411918067247381 .08389394240112462 22111.705065988568 44780.677  990177.1224792978 173486832 1   8.26208      -.03863   -.03939629 10.003862   -.03079537 -.031279564
"Argentina" 2020                  .                  .                  .                  .                  .         .                  .         . 1         .            .            .         .            .           .
"Barbados"  1970                  . 1746.1749097790491   36.0925944431108                  .                  .   238.896                  .  417154.2 2  7.465183            .            .         .            .           .
"Barbados"  1971                  .  1812.640313772739  32.50865355427918                  .                  .   240.094                  .  435204.1 2   7.50254    .03806343    .03735685         .            .           .
"Barbados"  1972                  . 1824.8544580391913 28.912677367223694                  .                  .   241.515                  .  440729.7 2  7.509255   .006738317   .006715775         .            .           .
"Barbados"  1973                  . 1961.8375560803497  29.91746812115526                  .                  .   243.069                  .  476861.9 2  7.581637    .07506522     .0723815         .            .           .
"Barbados"  1974                  .  1934.666568360079 27.474042841421063                  .                  .   244.644                  .  473304.6 2   7.56769  -.013849764  -.013946533         .            .           .
"Barbados"  1975                  .  2064.328511818957 22.397899954083872                  .                  .   246.154                  .  508142.7 2   7.63256   .067020305    .06486988         .            .           .
"Barbados"  1976                  . 2193.3099835707944 23.878226900860735                  .                  .   247.583                  . 543026.25 2  7.693167    .06248108    .06060696         .            .           .
"Barbados"  1977                  .  2549.614696250463  22.92236939200876                  .                  .   248.937                  .  634693.4 2  7.843698     .1624507    .15053034         .            .           .
"Barbados"  1978                  .  2699.826082617303 17.532559442717044                  .                  .   250.203                  .  675504.6 2  7.900943    .05891533    .05724525         .            .           .
"Barbados"  1979                  . 3311.6903230340045  18.79265496945313                  .                  .   251.351                  .  832396.7 2  8.105214      .226631     .2042713         .            .           .
"Barbados"  1980                  .  3533.121379271497 21.086997968077384                  .                  .   252.394                  .  891738.6 2  8.169937   .066863455   .064723015         .            .           .
"Barbados"  1981                  .  3534.566764282888 16.935711274311668                  .                  .   253.298                  .  895298.7 2  8.170346  .0004090958  .0004091263         .            .           .
"Barbados"  1982                  . 3496.3328068118567 15.649325134059419                  .                  .   254.082                  .  888355.3 2   8.15947  -.010817155  -.010876656         .            .           .
"Barbados"  1983                  . 3221.6962685852113 15.428764740211093                  .                  .   254.788                  .  820849.6 2  8.077663   -.07854988   -.08180618         .            .           .
"Barbados"  1984                  . 3298.6092505588767 15.951927876224845                  .                  .   255.497                  .  842784.8 2  8.101256    .02387344    .02359295         .            .           .
"Barbados"  1985                  .  3113.464139634753 15.304106702733872                  .                  .   256.263                  .  797865.7 2  8.043491   -.05612823   -.05776501         .            .           .
"Barbados"  1986                  .  4126.876466238938 14.829100269829537                  .                  .   257.107                  . 1061048.9 2  8.325276     .3254935      .281785         .            .           .
"Barbados"  1987                  .  4393.592392321903  9.392383536823406                  .                  .   258.019                  . 1133630.4 2  8.387902      .064629   .062625885         .            .           .
"Barbados"  1988                  .   4429.39489905491 10.098951168210057                  .                  .   258.977                  . 1147111.4 2  8.396018   .008148801   .008115768         .            .           .
"Barbados"  1989                  .  4696.996977905582  9.094687665133016                  .                  .   259.957                  . 1221017.3 2  8.454679    .06041504    .05866051         .            .           .
"Barbados"  1990 329.05870530120876  4601.567378824831 10.054792013325633 .12671621446899942 13984.031738691485   260.936  3648.937305767201 1200714.6 2  8.434153   -.02031715   -.02052593  9.545671            .           .
"Barbados"  1991 342.92892559613847 4591.4571920139515  9.222274001736615 .14290350411505676  13388.94694879496   261.905 3506.6321506241434 1202525.6 2  8.431952  -.002197118 -.0022001266  9.502185    -.0425546   -.0434866
"Barbados"  1992  322.4513310350723 4056.3279737940393  7.790549873503577 .11958463758493243 12579.659574588146   262.886 3307.0163869251796 1066351.9 2  8.308033   -.11654889   -.12391949 9.4398365   -.06044444  -.06234837
"Barbados"  1993 316.72566817628774  4000.862925692591  6.381924750964159 .12199718004297569 12631.950383843638   263.874 3333.2432755863565 1055723.8 2  8.294266   -.01367371  -.013767242  9.443985  .0041567744 .0041484833
"Barbados"  1994  347.9896186555122  4466.927426815008  8.370563793210959  .1301878145911489  12836.38128077891   264.893 3400.2675466093674 1183257.8 2  8.404456      .116491     .1101904  9.460039   .016183637  .016054153
"Barbados"  1995 354.34650122869004  4621.605251502314   6.50627716844551 .13642765791010547 13042.615731993916   265.955  3468.748867002442   1229139 2  8.438498    .03462734   .034041405  9.475977     .0160664  .015937805
"Barbados"  1996  357.8844175709426  4833.478499216316 6.7460555602325085 .13689268655382394 13505.696984580747   267.049 3606.6828740353035 1290775.6 2  8.483322    .04584408     .0448246  9.510867   .035505243  .034890175
"Barbados"  1997 388.71441493944394  5475.375086094262  6.110728479212351  .1448871869271294  14085.85551669661   268.187 3777.6433334563144 1468424.4 2  8.608016     .1328022    .12469387  9.552926    .04295658   .04205894
"Barbados"  1998   368.439698177946  5360.322434366408  6.704598344861634  .1357589067102381 14548.710306937453   269.336 3918.4914392293063 1443727.8 2   8.58678  -.021012744   -.02123642  9.585258   .032859545  .032331467
"Barbados"  1999  390.4973597982483 5676.7448698656635  4.208468576294624   .140323000553612 14537.217032142218   270.451 3931.6048835598954 1535281.4 2  8.644134    .05903048    .05735397  9.584467 -.0007899858 -.000790596
"Barbados"  2000  393.4087202990977  5949.997448884177    6.9144815584872 .14520248342305545 15124.213424553884   271.515  4106.450807967747 1615513.5 2  8.691146    .04813543    .04701233  9.624052    .04037887   .03958511
"Barbados"  2001  386.4669632657052  5686.119182373591  5.863251737097423 .14015779392676775 14713.079571731076     272.5  4009.314183296718 1549467.5 2  8.645783   -.04434931   -.04536247  9.596492   -.02718382 -.027560234
"Barbados"  2002    378.99366595302  5601.058001524491  5.916533649224166  .1372706263071984 14778.764144883615   273.425  4040.883586314803 1531469.3 2  8.630711  -.014959444  -.015072823  9.600946   .004464366 .0044546127
"Barbados"  2003  372.3589584275251  5604.193413521368   5.80739143278922  .1347312725353447 15050.513185416357   274.329  4128.792231642084 1537392.8 2   8.63127 .00055978925  .0005598068  9.619167   .018387806  .018220901
"Barbados"  2004 373.01291089856505  5673.532562351017  5.573228634055563 .13462577908432746 15210.016588122453   275.284  4187.074206444701 1561832.8 2  8.643567   .012372726   .012296677  9.629709   .010597872  .010541916
"Barbados"  2005 366.61274458646216  5774.953282950012  6.348339237943818 .13144121188599053  15752.18911023985   276.323  4352.692151508805 1595752.4 2  8.661285   .017876115   .017718315  9.664735   .035645753  .035025597
"Barbados"  2006 350.68787777041615  5825.842637155089 5.1105895389580045 .12427966196729479  16612.61482488162   277.477  4609.618523763677 1616537.4 2  8.670059   .008812081   .008773804  9.717917    .05462261    .0531826
"Barbados"  2007 348.25149803075647  5886.820201592582 5.5068252679140315 .12393129924800848  16903.93360798315     278.7  4711.126296544903 1640656.8 2  8.680471   .010466738   .010412216  9.735302   .017535998   .01738453
"Barbados"  2008 406.91695145747286  6895.887684926475  4.579264220402662  .1425634915918177 16946.670961303436   279.946  4744.152748933052   1930476 2   8.83868     .1714113    .15820885  9.737826   .002528249  .002524376
"Barbados"  2009 415.27768260765373  6652.700964721799  5.002219547343357 .14586809987088573 16019.885593050618   281.104    4503.2539197489   1870101 2  8.802778   -.03526547  -.035902023  9.681586   -.05468835  -.05624008
"Barbados"  2010   394.785943969982  6157.325090599376  4.187971216003144  .1376868297746992 15596.616811330943   282.131   4400.28909759761 1737172.3 2  8.725398   -.07446237   -.07738018  9.654809   -.02642146  -.02677727
"Barbados"  2011  419.4301935737128  6477.992939013307 3.8312669791159486 .14552383638899605 15444.746320760123   282.987  4370.662427072945 1833187.8 2  8.776166    .05207909     .0507679  9.645024    -.0097374 -.009784698
"Barbados"  2012 395.64174416796095  6067.437508422563  3.747293928404336 .13720037104298788 15335.685877086737   283.701  4350.749419015385   1721338 2  8.710691   -.06337695   -.06547451  9.637938   -.00706133 -.007086754
"Barbados"  2013  406.3463344659614  6131.089703791145  2.664156915859806 .13975465156940423 15088.335205112406   284.296 4289.5533454726365 1743044.3 2 8.7211275   .010490787   .010436058  9.621677  -.016129091 -.016260147
"Barbados"  2014  370.8090910116841  5577.525661240319 2.8709489833484527 .12568224256510316 15041.501938431637   284.826   4284.21083111573 1588624.4 2    8.6265   -.09028804   -.09462738  9.618568  -.003103939 -.003108978
"Barbados"  2015 358.61963275242823  5516.619785255851 1.6810256186963153 .12087318459526923   15382.9274290296   285.324  4389.118385760442   1574024 2 8.6155205  -.010919874  -.010979652  9.641013   .022698896  .022444725
"Barbados"  2016  328.6869692153725  5172.937576584527  2.204469305036939 .11166096345783456 15738.188796876071   285.796  4497.911405391994   1478405 2  8.551196   -.06229942   -.06432438  9.663845   .023094524  .022831917
"Barbados"  2017  336.3728054866606  5310.948407000248  2.799368628679885 .11571576270308377 15788.875677141688   286.232   4519.28146281962 1520163.4 2  8.577526   .026679393   .026329994  9.667061    .00322063   .00321579
"Barbados"  2018 343.03007650511717  5376.944869419369 3.5510648345994404 .11634852461873475 15674.849634764192   286.641  4493.054574158443   1541253 2  8.589875   .012426493    .01234913  9.659813  -.007221923 -.007247925
end
label values pais pais
label def pais 1 "Argentina", modify
label def pais 2 "Barbados", modify

Can be generate index number easier than using the next form:

bys Country (year): egen base_co2pc=total((year==2000)*co2pc)
bys Country (year): gen co2pc_index=100*co2pc/base_co2pc
bys Country (year): egen base_gdppc=total((year==2000)*gdppc)
bys Country (year): gen gdppc_index=100*gdppc/base_gdppc

To generate co2pc and gdppc growth i do:

foreach x of var co2pc gdppc { 
    gen ln_`x' = ln(`x')
    gen `x'_gr = D.`x'/L.`x'
    gen ln_`x'_gr = D.ln_`x'
}

Any help is very much appreciated.

Thanks,

Sebastián

[Clyde Schechter]

I'm not sure what you're asking. The code you show for calculating the index takes only four lines, is quite transparent, and does the job correctly. How were you hoping to improve upon that? One could combine the treatment of co2pc and gdppc into a loop, but the code would still be four lines (including the looping code), wouldn't run any faster, and would be substantially less transparent. Did you have something else in mind?

[Sebastian Kruk]

I thought I can do it in an simple way because data are currently xtset.

Is

gen `x'_gr = D.`x'/L.`x'

equivalent

bys pais (year): gen `x'_gr = D.`x'/L.`x'

when data are xtset?

[Clyde Schechter]

That works for generating year on year growth. But for an index anchored at a specific year, there is nothing quite analogous. I could concoct a sequence of commands that uses lag operators after setting an initial value when year == 2000 to fill in the index from 2000 forward. But that code would not work for years earlier than 2000, and code using the lag and forward operators to try to do that would fail. Instead it would require perhaps creating a "backwards year" variable, -xtset-ing with that and then using the lag operators on this backwards-year sorted data. The resulting code would be longer than what you wrote, would require longer to execute (noticeably so if the data set is large), and would be utterly opaque. It really has nothing to recommend it.

[Sebastian Kruk]

I mean If I use

 
 gen `x'_gr = D.`x'/L.`x'

I will generate growth rate by country?

How I can generate a growth rate by country?

Thanks for your help!

[Clyde Schechter]

If you -xtset country year-, then, yes, -gen `x'_gr = D.`x'/L.`x'- will, ineed, generate growth rate by country.

[Sebastian Kruk]

Thank you Clyde!