calculate averages of groups of observations separated by groups of missing observations

[Sam Volpe]

Hi everybody,

I have panel data with a few variables that have a somewhat random pattern. I need to generate summary statistics for each chunk of data in the varlist: accfrag decfrag maintfrag crisisfrag miraclefrag stablefrag stagfrag. As you can see, there's nothing really common between these chunks of data, not even number of observations.

Each chunk has a number of observations equal to the corresponding period length plus 1.

I'm wondering if anybody has any ideas for how to go about this problem?
Thank you!

* Example generated by -dataex-. For more info, type help dataex
clear
input float(growth_regime periodlength frag_index accfrag decfrag maintfrag crisisfrag miraclefrag stablefrag stagfrag)
4 13 1.1493217         .         .         . 1.1493217         .         .         .
4 13 1.1500545         .         .         . 1.1500545         .         .         .
4 13 1.1507871         .         .         . 1.1507871         .         .         .
4 13 1.1518463         .         .         . 1.1518463         .         .         .
4 13 1.1521727         .         .         . 1.1521727         .         .         .
4 13 1.1529053         .         .         . 1.1529053         .         .         .
4 13 1.1522527         .         .         . 1.1522527         .         .         .
4 13 1.2388158         .         .         . 1.2388158         .         .         .
4 13  1.239795         .         .         .  1.239795         .         .         .
4 13 1.1879622         .         .         . 1.1879622         .         .         .
4 13 1.1899204         .         .         . 1.1899204         .         .         .
4 13  1.127124         .         .         .  1.127124         .         .         .
4 13 1.1236141 1.1236141         .         . 1.1236141         .         .         .
4 13 1.1281831 1.1281831         .         . 1.1281831 1.1281831         .         .
1 13  1.130794  1.130794         .         .         .  1.130794         .         .
1 13  1.133485  1.133485         .         .         .  1.133485         .         .
1 13 1.1338112 1.1338112         .         .         . 1.1338112         .         .
1 13 1.1316068 1.1316068         .         .         . 1.1316068         .         .
1 13 1.1369085 1.1369085         .         .         . 1.1369085         .         .
1 13 1.1416377 1.1416377         .         .         . 1.1416377         .         .
1 13 1.0815322 1.0815322         .         .         . 1.0815322         .         .
1 13  .9746989  .9746989         .         .         .  .9746989         .         .
1 13 .56646764 .56646764         .         .         . .56646764         .         .
1 13 .56915855 .56915855         .         .         . .56915855         .         .
1 13  .5735613  .5735613         .         .         .  .5735613         .         .
1 13 .57731134 .57731134         .         .         . .57731134         .         .
1 13  .5823668  .5823668  .5823668         .         .  .5823668         .         .
1 13  .5867696         .  .5867696         .  .5867696  .5867696         .         .
4 10  .5912523         .  .5912523         .  .5912523         .         .         .
4 10 .53220606         . .53220606         . .53220606         .         .         .
4 10   .492501         .   .492501         .   .492501         .         .         .
4 10  .4969837         .  .4969837         .  .4969837         .         .         .
4 10  .5011401         .  .5011401         .  .5011401         .         .         .
4 10 .50568944         . .50568944         . .50568944         .         .         .
4 10 .50937295         . .50937295         . .50937295         .         .         .
4 10  .5537676         .  .5537676         .  .5537676         .         .         .
4 10  .4810795         .  .4810795         .  .4810795         .         .         .
4 10  .3022385         .  .3022385         .  .3022385         .         .         .
4 10 .37884295         . .37884295         . .37884295         .         .         .
4  5  .4470737         .         .         .  .4470737         .         .         .
4  5  .3963463         .         .         .  .3963463         .         .         .
4  5  .7526202         .         .         .  .7526202         .         .         .
4  5  .6688801         .         .         .  .6688801         .         .         .
4  5   .770782   .770782         .         .   .770782         .         .         .
4  5  .7729334  .7729334         .         .  .7729334  .7729334         .         .
1 12  .7190617  .7190617         .         .         .  .7190617         .         .
1 12  .7223123  .7223123         .         .         .  .7223123         .         .
1 12  .7262556  .7262556         .         .         .  .7262556         .         .
1 12  .7307451  .7307451         .         .         .  .7307451         .         .
1 12   .734835   .734835         .         .         .   .734835         .         .
1 12   .794482   .794482         .         .         .   .794482         .         .
1 12   .796647   .796647         .         .         .   .796647         .         .
1 12   .799498   .799498         .         .         .   .799498         .         .
1 12  .8018028  .8018028         .         .         .  .8018028         .         .
1 12  .8052664  .8052664         .         .         .  .8052664         .         .
1 12  .7496784  .7496784         .         .         .  .7496784         .         .
1 12  .6955136  .6955136  .6955136         .         .  .6955136         .         .
1 12  .6978116         .  .6978116         .         .  .6978116         .  .6978116
3  9  .6996368         .  .6996368         .         .         .         .  .6996368
3  9  .7013153         .  .7013153         .         .         .         .  .7013153
3  9   .703247         .   .703247         .         .         .         .   .703247
3  9  .6681315         .  .6681315         .         .         .         .  .6681315
3  9  .6695903         .  .6695903         .         .         .         .  .6695903
3  9  .6709757         .  .6709757         .         .         .         .  .6709757
3  9  .7688444         .  .7688444         .         .         .         .  .7688444
3  9  .7698235         .  .7698235         .         .         .         .  .7698235
3  9  .6767297         .  .6767297         .         .         .         .  .6767297
3  9  .6840762         .  .6840762         .         .         .         .  .6840762
4  5 .52876216         .         .         . .52876216         .         .         .
4  5 .52938837         .         .         . .52938837         .         .         .
4  5  .5296149         .         .         .  .5296149         .         .         .
4  5 .52994806         .         .         . .52994806         .         .         .
4  5 .53151333 .53151333         .         . .53151333         .         .         .
4  5  .5303212  .5303212         .         .  .5303212         .  .5303212         .
2  6  .5293088  .5293088         .         .         .         .  .5293088         .
2  6  .5292023  .5292023         .         .         .         .  .5292023         .
2  6 .52927554 .52927554         .         .         .         . .52927554         .
2  6 .58570516 .58570516         .         .         .         . .58570516         .
2  6  .5858517  .5858517         .         .         .         .  .5858517         .
2  6 .58596504 .58596504         .         .         .         . .58596504         .
2  6   .587204         .         .   .587204         .         .   .587204   .587204
3 25  .5886627         .         .   .590268         .         .         .  .5886627
3 25   .590268         .         .  .5918733         .         .         .   .590268
3 25  .5918733         .         . .59333205         .         .         .  .5918733
3 25 .59333205         .         . .59504384         .         .         . .59333205
3 25 .59504384         .         .  .4296117         .         .         . .59504384
3 25  .4296117         .         .  .5564309         .         .         .  .4296117
3 25  .5564309         .         .  .3337166         .         .         .  .5564309
3 25  .3337166         .         .   .241985         .         .         .  .3337166
3 25   .241985         .         .  .2422048         .         .         .   .241985
3 25  .2422048         .         . .24307735         .         .         .  .2422048
3 25 .24307735         .         .  .2455817         .         .         . .24307735
3 25  .2455817         .         .  .2477597         .         .         .  .2455817
3 25  .2477597         .         . .25073698         .         .         .  .2477597
3 25 .25073698         .         . .25298825         .         .         . .25073698
3 25 .25298825         .         . .25563914         .         .         . .25298825
3 25 .25563914         .         .  .4839083         .         .         . .25563914
3 25  .4839083         .         .  .5924911         .         .         .  .4839083
3 25  .5924911         .         .  .5934369         .         .         .  .5924911
3 25  .5934369         .         .  .5943827         .         .         .  .5934369
end
label values growth_regime gr_regimes_labels
label def gr_regimes_labels 1 "Miracle", modify
label def gr_regimes_labels 2 "Stable", modify
label def gr_regimes_labels 3 "Stagnant", modify
label def gr_regimes_labels 4 "Crisis", modify

[Fei Wang]

Display averages of each chunk for every variable.

foreach v of varlist accfrag-stagfrag {
    gen `v'_chunk = sum(!mi(`v') & mi(`v'[_n-1]))
    tabstat `v' if `v'_chunk > 0, by(`v'_chunk) not
}

Summary for variables: accfrag
     by categories of: accfrag_chunk 

accfrag_chunk |      mean
--------------+----------
            1 |  .9323425
            2 |  .7585581
            3 |  .5508929
-------------------------

Summary for variables: decfrag
     by categories of: decfrag_chunk 

decfrag_chunk |      mean
--------------+----------
            1 |  .5010931
            2 |  .7004746
-------------------------

Summary for variables: maintfrag
     by categories of: maintfrag_chunk 

maintfrag_chunk |      mean
----------------+----------
              1 |  .4260836
---------------------------

Summary for variables: crisisfrag
     by categories of: crisisfrag_chunk 

crisisfrag_chunk |      mean
-----------------+----------
               1 |  1.163911
               2 |  .5411377
               3 |  .5299247
----------------------------

Summary for variables: miraclefrag
     by categories of: miraclefrag_chunk 

miraclefrag_chunk |      mean
------------------+----------
                1 |  .8965529
                2 |  .7533459
-----------------------------

Summary for variables: stablefrag
     by categories of: stablefrag_chunk 

stablefrag_chunk |      mean
-----------------+----------
               1 |  .5578542
----------------------------

Summary for variables: stagfrag
     by categories of: stagfrag_chunk 

stagfrag_chunk |      mean
---------------+----------
             1 |  .7009256
             2 |  .4257976
--------------------------

[Sam Volpe]

Fei Wang Thank you very much for your help! This worked! Much appreciated