Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • a matching and test problem with weird result

    Hi all
    I'm recently doing a empirical study need to match case sample with one or more potential benchmark samples using a specific measure and see if there is an statistically significant effect of the given treatment to the case firms, all the data are observational ones.

    after I matched the cases with controls using , say inventory, I want to test whether the match is or not, So I use paired t test on inventory of the matched pairs, but the testing result show that they are significantly different, I went to draw a plot of paired inventory and they seems like almost perfectly matched.

    I checked out several times and still don't know why ,really need your help . below is my code and data

    Code:
    clear
    set more off
    use "E:\Research\privatization\match_merge-1.dta"
    sort firmid year firmid_U year_U
    bys firmid: keep if year ==year_U 
    local COVARIATES IR_U ROA_U totalasset_U sales_U Leverage_U Export_U
    foreach i of local COVARIATES {
        bys firmid: egen `i'_control_mean = mean(`i')
    }
    duplicates drop firmid, force
    
    
    
    local COVARIATES IR ROA totalasset sales Leverage Export
    foreach i of local COVARIATES {
        bys firmid: gen dif_`i'_control_mean = `i' - `i'_U_control_mean
        ttest dif_`i'_control_mean == 0
        signrank dif_`i'_control_mean = 0
    }
    keep Export_U_control_mean /// 
     Leverage_U_control_mean sales_U_control_mean totalasset_U_control_mean ROA_U_control_mean ///
     IR_U_control_mean ROA Leverage Export IR totalasset sales firmid year
    see the attachment for data the variable I matched on is IR, and all variables with "U" are from the matched firms(control) others are are belong to cases. the difference between IR and IR_U should be very small, at least no statistically significant since each IR_U is within (0.9- 1.1)*IR, this is how the matching process performed any way.
    Code:
    * Example generated by -dataex-. To install: ssc install dataex
    clear
    input int year str9 firmid float(ir roa ir_u_control_mean roa_u_control_mean)
    2003 "232327184"  .00412  -.00484  .00412  .00305
    2003 "706673565"  .00412   .23336  .00412  .00305
    2002 "216528971"  .00526   .04483  .00526  .02024
    2002 "182717706"  .00526  -.00117  .00527   .0107
    2002 "705737504"  .00573   .08984  .00526  .05358
    2001 "HB1896279"   .0066   .10601   .0066  .12415
    2001 "72665285X"   .0066   .05568   .0066  .00219
    2001 "72520107X"   .0066   .00285   .0066  .03415
    2001 "726318448"   .0066    .0041   .0066       0
    2001 "105220528"   .0066   .19845   .0066  .00219
    2001 "717965694"   .0066   .11833   .0066 -.07772
    2003 "169598212"  .00879   .00376   .0086 -.01168
    2003 "20636183X"  .00917   .03289  .00871  .01456
    2000 "166810804"  .01178   .10668  .01178  .01431
    2000 "164962652"  .01178   .06257  .01178   .0176
    1999 "190332195"  .01375  -.10828   .0138  .02257
    1999 "919660637"  .01375   .01411  .01375  .00995
    1999 "715887098"  .01375   .15881  .01375  .00995
    1999 "194948667"  .01375   .03594   .0138  .02257
    2001 "731480669"   .0154  -.02748  .01633  .01339
    2003 "168050714"  .01692   .21572  .01541  .00679
    2003 "615244124"  .01904   .00014  .01998  .08153
    1998 "192627528"  .01916   .15385  .01916 -.02127
    1998 "626582460"  .01916   .02001  .01916  .07178
    1998 "180361507"  .01916   .06744  .01926  .01506
    1998 "170076104"  .01916   .00399  .01916  .00083
    1998 "215612127"  .01916   .07416  .01916  .00374
    1998 "116540009"  .01916  -.00867  .01916  .00191
    1998 "14709083X"  .01916  -.12689  .01925  .01519
    1998 "132630709"  .01916   .00377  .01916  .01367
    1998 "616574205"  .01916   .00646  .01925  .00011
    2001 "727757645"  .02006    .0001  .01915  .05457
    2002 "143048124"   .0201   .03973  .01923  .04959
    1998 "229211965"  .02276   .08784  .02226  .00844
    2001 "205170222"  .02283   .00198   .0229  .07606
    2002 "16990403X"  .02294   .00148  .02214 -.00265
    1999 "194279248"  .02394   .01678    .025  -.0353
    2000 "JL272201X"  .02405    .0874  .02381  .00015
    2003 "13685155X"  .02647        0  .02545 -.00077
    2003 "737219103"  .02683   .12518  .02589  .11282
    1998 "103825937"  .02781   .15385  .02655 -.12689
    2000 "140030965"  .02795     .037  .02769  .00609
    1998 "103665566"  .02813  -.06618   .0273  .00301
    1998 "187881396"  .03099   .00008  .03165 -.00657
    2000 "230001797"  .03294   .01906  .03271  .00247
    2000 "164961094"  .03304   .01856  .03225       0
    1999 "184563536"  .03324  -.02559  .03493   .0948
    1999 "170598244"   .0339   .09995  .03289  .15881
    1998 "102867537"  .03438   .03295   .0348  .06478
    1998 "132741687"  .03742   .00055  .03655  .00084
    2001 "176762113"  .03866   .00003  .03805    .026
    2000 "71212989X"   .0392   .00798  .03636  .00372
    2003 "101610457"  .03982   .05922  .03814  -.0444
    2000 "124831009"     .04   .06649  .03927  .02544
    1999 "132701319"  .04132  -.02609  .04202  .01607
    1999 "225420129"  .04137  -.04074  .04295  .02143
    1999 "148269946"  .04186   .00429  .04211   .0151
    1999 "169891785"  .04191    .0012  .04226  .01826
    1998 "24386978X"  .04239   .15385  .04318  .00223
    2000 "203654282"  .04357   .00076  .04655  .02601
    1999 "161123570"  .04448   .02972  .04406   .0236
    2000 "186201651"  .04459    .0212  .04844  .01465
    1998 "VS3828986"  .04481   .00021  .04529   .0154
    2003 "742070032"  .04557   .07756  .04492  .02549
    2002 "734718142"  .04668  -.06094  .04652 -.00421
    2000 "617792714"  .04747   .00024  .04916  .01858
    2000 "621881808"  .04809   .00603   .0485  .05168
    1998 "218610784"  .04819   .09777  .04659  .00898
    2003 "735157172"  .04962   .04103  .04915  .00373
    2003 "175743295"  .04978  -.09189  .05318  .00635
    1998 "133585989"  .05045    .0031  .05102  .00252
    2003 "737611922"   .0505  -.00121  .04977  .04711
    2003 "749785153"  .05208  -.09189  .04977  .04711
    2001 "141850076"  .05243  -.05508  .05045  .04249
    1999 "713133823"  .05267   .06548  .05058  .00892
    2002 "715595191"  .05267   .06324  .04889  -.0034
    1998 "109630233"  .05271   .02656  .05491  .00487
    2002 "16500945X"  .05334   .00723  .05074 -.00243
    2001 "729301289"  .05353  -.05229  .05218  .00889
    1999 "706958317"  .05384   .01272    .052  .02231
    2002 "728321392"  .05405    .0024  .05347   .0456
    1999 "14202995X"  .05421   .00225  .05598 -.00336
    1999 "142871813"  .05497   .00013  .05598 -.00336
    2000 "111952055"  .05556        0  .05515  .04795
    2002 "128516356"  .05565  -.01339  .05347   .0456
    1998 "110053517"  .05708   .00053  .05501  .15385
    1999 "190410093"  .05743   .05761  .05687 -.09634
    1999 "700799669"  .05832   .12045  .05963  .02182
    1999 "166560016"  .05883   .01409  .06072  .06395
    1999 "146110050"  .05952   .02464  .05943  .00188
    1998 "22594028X"  .06162        0  .06275 -.00362
    2003 "608470190"  .06238        0  .06486  .23336
    2003 "157620646"  .06354   .06418  .06729  .07609
    2000 "706256817"  .06644   .00866  .07297 -.08035
    2001 "103607112"  .06721   .01439  .06344  .00049
    1999 "146836185"  .07116   .03668  .07058  .00483
    1998 "101105573"   .0712   .01934  .07242  .00253
    1999 "140133462"  .07139  -.10828  .07058  .00483
    1998 "181870231"  .07228   .11452  .07095  .02296
    2002 "17558188X"  .07267   .00781  .07251  .02382
    2002 "134878652"  .07377   .05705  .07416   .0279
    2001 "733897759"  .07515   .00015  .07765   .0046
    1998 "102474549"   .0754   .04524  .07553  .00954
    2003 "740989678"  .07551   .07795  .07343  .02391
    2001 "128722387"   .0768   .00702   .0751  .01185
    2003 "173072774"  .07726   .06045  .07838  .01939
    2002 "193126952"  .07747  -.02237  .07808  .02799
    2001 "149042508"  .07785   .05293  .07719 -.00882
    2001 "10808420X"  .07816    .0515  .08013  .02748
    2002 "166640833"  .07821   .00283  .07834 -.02095
    2002 "192269097"  .07871   .02489  .07834 -.02095
    1998 "HU0815595"  .07875   .08587  .07814  .01695
    2001 "13932208X"  .07886   .03416  .07765   .0046
    2002 "719832297"  .07964   .12578  .08701 -.05843
    1998 "15282202X"     .08   .00597  .08037 -.00207
    2003 "218170344"  .08096  -.09189  .08161  .01859
    2002 "148800602"  .08112  -.06221  .08701 -.05843
    2003 "149341030"  .08144  -.09189  .08161  .01859
    1999 "165684842"  .08295   .03548  .08473  .00456
    1999 "103293555"  .08299     .003   .0818 -.00516
    2000 "145711751"  .08388  -.02451  .08793 -.00205
    2000 "217310288"  .08406   .00088  .08478  .04162
    2002 "744675529"  .08438   .06997  .08744 -.01247
    2002 "727934373"  .08492   .02602  .08783  .00919
    2002 "715178275"  .08527   .01542   .0908  .02304
    2001 "152384787"  .08531   .00137  .08713 -.01297
    1998 "17464165X"  .08602   .02923   .0828  .09767
    2000 "10714985X"  .08611   .03398  .08873  .00117
    2003 "104794941"  .08652   .03742  .08765  .02074
    1999 "140943051"  .08667   .00786  .08923 -.00974
    2000 "142225264"  .08737   .00546  .08839  .04772
    1998 "217713618"  .08743   .15385  .08929  .02454
    2000 "166730791"  .08792   .07753  .08554  .00126
    1998 "168453433"  .08796        0  .08771  .00281
    1999 "209700470"  .08819  -.10414  .09048  -.0079
    2002 "138297206"  .08845   .00609  .08772  .00408
    1998 "215552806"  .08907  -.07818  .08684 -.00843
    2000 "142833200"  .08948  -.00995  .08977  .05553
    1998 "217670937"  .09104   .01352  .08931  .01016
    2003 "709043657"  .09263   .09042  .10112   .0018
    2002 "743670996"  .09286  -.03792  .09527  .07914
    2002 "157870709"  .09308   .09533  .09169 -.00637
    2000 "169418592"  .09401   .06824  .09115  .00838
    1998 "178764964"  .09534   .01263  .09355  .02457
    2003 "470110503"  .09572    .0979  .09539  .03529
    2003 "735000714"   .0964  -.05181  .09791  .02156
    1998 "631144510"  .09727  -.08441  .09521   .0611
    2001 "210400993"  .09753  -.00407  .09755  .00217
    2001 "200321431"  .09775    .0061  .09755  .00217
    2002 "156504379"  .09817   .05356  .09797  .00696
    2001 "109411516"  .10008   .00379  .09999  .02323
    2000 "166827411"   .1009   .02314  .09864  .00551
    2003 "17961196X"  .10151        0  .09862  .06142
    1999 "132963888"  .10159   .00008  .10266  .00902
    1998 "198052454"   .1026   .06729  .10023  .02138
    2002 "738995534"  .10295   .02294  .10294  .04045
    2003 "707331508"  .10305  -.00702  .10221  .09082
    1998 "802590751"  .10375   .15385  .10358  .15385
    1998 "178120683"  .10466   .01495   .1059  .05328
    1999 "250817929"  .10521    .0343   .1055  .06602
    1999 "136360718"  .10599   .00868  .09787  .15881
    2001 "216820833"  .10599   .00048  .10987  .04415
    2003 "104625523"  .10758   .00548  .10611  .02223
    2002 "20790797X"  .10764  -.02479   .1068 -.02095
    2000 "168132031"  .10917   .07552  .10945  .05622
    2001 "727980935"  .10942   .00182  .11302  .03076
    1999 "191430539"  .10954   -.0071  .10922  .01517
    1998 "133031468"  .11029   .01833  .11009  .01665
    2002 "198074063"  .11039   .01298  .11151  .02652
    1999 "192770658"  .11262   .01902  .11377  .01231
    1998 "137701473"  .11291   .00277  .11236 -.00778
    2003 "174070332"  .11354   .12718  .11695  .03024
    2003 "173318017"  .11371   .02996  .10936  .02418
    1998 "123716682"  .11523   .01109  .12141 -.04929
    2002 "104770130"   .1164    .0175  .12048  .01421
    2000 "198096780"   .1168   .00305  .11658  .00893
    1999 "143109708"  .11685    .0657    .114   .0181
    2001 "210500556"  .11834   .01996  .11522  .03412
    1999 "144280451"  .11919   .02969  .12057   .0109
    2003 "203142308"  .11957   .00517  .11612   .0052
    1998 "198085299"  .12067   .00422  .12713  .04024
    2003 "16358790X"  .12103   .06148   .1214  .04529
    2000 "103609732"  .12203  -.01591  .12393   .0303
    2002 "16356787X"  .12289   .01822  .11925  .00601
    2000 "715809129"   .1235  -.00349  .12136  .02312
    2000 "601139796"  .12355   .17716  .12136  .02312
    1999 "144590759"  .12378   .00443   .1302 -.00995
    1999 "136927120"  .12455        0  .13416  .15881
    1998 "299201885"  .12467   .07107  .12303 -.00026
    2003 "167279170"  .12535   .04071  .12834  .00066
    2003 "190649133"  .12555    .0678  .12382  .05123
    1998 "190668350"  .12557   .08288  .12937 -.02772
    1998 "203350756"  .12557  -.00678  .12225   .0162
    2001 "148953174"  .12826  -.05826  .12476  .00886
    2003 "198605178"  .12889   .12361  .12735  .03809
    2002 "133737591"   .1297  -.04676  .12543  .01982
    2003 "745148851"   .1297  -.02758  .12427  .00123
    2000 "165147684"  .13002   -.0092  .12999 -.01186
    2002 "166810601"  .13017   .00816   .1345  .01447
    2000 "201935195"   .1304   .00305  .13083  .05797
    2003 "201974661"  .13111  -.02558  .13265   .0309
    2000 "190659796"  .13176   .01741  .13154  .00269
    2003 "151104026"  .13192   .02175  .13265   .0309
    2000 "143057194"   .1328   .00157  .13333 -.00986
    1999 "144844421"  .13384  -.03434  .13882 -.00209
    2002 "630999840"  .13392   .00341  .12739  .02557
    2000 "703828939"  .13423   .01351  .13311  .03117
    2001 "142508408"  .13438    .0508  .13474  .00851
    1998 "134419894"  .13598  -.00859    .135  .02805
    1998 "102952416"  .13599        0  .13335 -.00589
    1999 "70885110X"  .13722   .01182  .13678 -.00057
    2003 "185280650"  .13839        0  .13871  .03457
    1998 "21135225X"  .13963   .00372  .13721  .03222
    1998 "174681176"   .1401   .14552   .1392  .06694
    2000 "147048463"  .14025   .00693  .13851  .00274
    2003 "737619131"  .14062   .00736  .14557   .0789
    2001 "219520246"  .14096   .00267  .13804  .01343
    2000 "163526664"  .14141    .0097  .14404  .04153
    1998 "110765055"  .14192   .03151  .14289 -.01211
    2002 "72488194X"  .14266   .14365  .13731  .01385
    2001 "183843501"  .14327   .00824  .14588  .01681
    1998 "113242372"  .14391   .13158  .14849   .0006
    2001 "132841274"  .14414   .00812  .14633 -.03258
    2000 "137700059"  .14495   .00162  .14448  .04093
    2000 "140040936"  .14629    .0998  .15537 -.04789
    2003 "709413777"  .14648   .09015  .14764  .00278
    2001 "173790055"   .1468   -.0207  .14652 -.00849
    2002 "165921932"  .14694   .00005  .15274  .00662
    2000 "62114607X"  .14708    .0273  .14717  .04646
    1999 "133614399"  .14739  -.06748  .14014  .10532
    2000 "804415121"  .14756   .15711  .14796  .02405
    1999 "139884111"  .14787   .00255  .15017 -.00176
    2000 "202538183"  .14814   .00069  .15159   .0354
    1998 "125153334"  .14876    .0005   .1541  .04508
    2003 "120067998"  .14901  -.09189  .14764  .00278
    1999 "142030010"  .14903  -.04579  .14741  .00524
    1998 "214150661"   .1493  -.12689  .14856  .00664
    1998 "194134516"  .14987  -.07743  .15753  .01754
    1999 "631271923"  .15007  -.04043  .14963 -.00686
    1999 "205410150"  .15104   .00688  .15234 -.00423
    1999 "207051761"  .15124   .00041  .14857 -.00233
    2001 "AB2244175"  .15209   .00024  .14772  .00531
    2002 "249700686"  .15217   .06475  .15533  .03705
    2000 "138130764"  .15298   .00738  .15512  .02349
    2001 "154988218"  .15302  -.04786  .14601  .02228
    2002 "621540026"  .15334    .0001  .15517  .02904
    1998 "121130426"  .15421   .00602  .14926 -.01301
    2003 "133215849"   .1544   .00076  .15328  .00015
    1999 "137045226"   .1562   .02263   .1564  .00023
    2002 "103706639"  .15645   .13831  .15367  .03175
    2003 "134872859"  .15662     .047  .16191  .05566
    1999 "231113213"  .15696   .01492  .15667  .03865
    2001 "216576682"   .1577   .00197   .1518  .00381
    2003 "105282489"  .15789  -.00741  .14764  .00278
    2003 "198606509"  .16014   .05091  .16215  .00483
    2000 "165446959"  .16042  -.01437  .15927  .02536
    2001 "241562506"  .16049   .00737   .1518  .00381
    1998 "155597972"  .16109    .0011  .16524  .00709
    1998 "222480660"  .16126   .00029  .16384 -.02592
    1998 "131721019"  .16203   .15385  .16037 -.00769
    1998 "190462096"  .16277   .00435  .16125   .0079
    2003 "211350406"  .16281  -.09189  .16129  .02554
    1998 "144738100"  .16336   .01276   .1659 -.02483
    1999 "181614713"  .16337   .01116  .16152  .00312
    1999 "713712606"  .16416   .03971  .16152  .00312
    1998 "133747458"  .16431   .00996   .1655  .01662
    1999 "178784201"   .1647    .0391   .1673  .01792
    2001 "706311695"  .16538   .07877  .16468  .01932
    2003 "169886003"  .16673        0  .15466  .02049
    2003 "717611079"  .16678   .03515  .16507  .04221
    2002 "192175330"  .16745  -.00111  .17303  .02271
    1998 "107801131"  .16912   .00397  .16524  .00709
    2000 "158143319"   .1701   .00365  .17872  .01838
    2000 "22460249X"  .17048   .10357  .17602  .06838
    2000 "203500881"  .17068   .00304  .17122  .01348
    1999 "191429001"  .17102   .00246  .16879  .04762
    2003 "722037980"  .17166   .00788  .17945  .03532
    2000 "704614174"  .17167  -.02972  .17225  .00791
    2003 "709539214"  .17307   .01065  .16988  .03181
    2002 "727067011"  .17321   .02312  .17847  .05746
    1999 "136007447"  .17328   .01277  .17027  .00652
    2003 "107230412"  .17355  -.06761  .16973  .05287
    1999 "183561782"  .17405   .01978  .17514  .02045
    1999 "630890618"  .17418   .01463  .17627  .04294
    2003 "706067063"  .17468   .09328  .17863  .05616
    2000 "178426287"   .1757  -.00824  .17523    .007
    1998 "142525136"  .17628   .00023  .16954  .00227
    1998 "155583917"  .17655   .01474  .17555  .00888
    2000 "137585011"  .17663   .00165  .17254  .01215
    1999 "195877071"  .17833   .01024  .16774  .01361
    2000 "21857324X"  .17872  -.09425   .1773   .0319
    2002 "166603768"  .18129   .01751  .16452  -.0127
    2003 "214951477"  .18155   .03644  .18394  .02876
    2000 "704614289"   .1818   .09629  .17886  .01076
    2003 "722200450"  .18197  -.03107  .16418  .01397
    2003 "181312117"  .18255  -.00403  .18187  .03825
    2000 "165920550"  .18257   .01737  .18758  .00808
    2001 "700918920"  .18288  -.10077  .18473 -.03046
    1998 "205419024"  .18339    .0004  .18515 -.01243
    2003 "702882917"  .18369   .12767  .19061  .00015
    1998 "166810265"  .18374   .05603  .18947  -.0158
    2000 "17878421X"  .18443   .05822  .18537 -.00634
    1998 "163408115"  .18467   .15385  .18343  .07054
    2001 "125533775"   .1856   .09227  .18153 -.02388
    1999 "250816627"  .18581   .00377  .16973   .0086
    2002 "20550320X"  .18661   .00015  .18187  .00746
    1998 "101675058"  .18715   .10437  .18612 -.02821
    1998 "148481138"  .18715   .02829  .18622  .02791
    2000 "168729970"  .18729   .01341  .18008  .00534
    2000 "176405423"  .18999        0  .18978   .0509
    2000 "108613212"  .19058        0  .18763   .0034
    1998 "166810011"  .19068        0  .19036 -.02831
    2003 "21478295X"   .1911   .01272   .1911  .02335
    2002 "702220163"  .19198   -.0332  .18743  .00369
    2000 "833169775"  .19367   .17716  .18978   .0509
    2003 "22866409X"  .19386  -.09189  .20067  .01774
    2000 "138050000"  .19494    .0859  .18952 -.00045
    2000 "142556602"    .196   .01232  .19588  .00056
    1998 "241283929"  .19658   .02566  .19791  .00286
    1998 "198862705"  .19681   .02335  .19299  .03555
    1998 "134051348"  .19696   .07948  .20615 -.02069
    2003 "140130720"  .19841  -.09189  .20374  .13088
    2001 "194830466"  .19874   .16891  .19895  .01542
    1998 "21859362X"   .1989  -.00111  .19888 -.00215
    2002 "103855490"  .19892        0  .19276  .00648
    2001 "732846263"  .19921   .02498   .1941  .00393
    2002 "188322289"  .19977   .02284  .20283  .00127
    2000 "143671901"  .19978   .02857   .2066   .0148
    2003 "X03669965"  .20011   .00407   .1859  .01575
    1999 "142912996"  .20082   .00007  .20231  .02526
    2001 "175041939"  .20277  -.06546  .20578   .0039
    1999 "115910194"   .2031   .02811  .19559  .01505
    1999 "18011416X"  .20368   .00244  .21879  .02133
    1999 "132622186"  .20406   .00042  .20224  .02528
    2001 "146229086"  .20495  -.01577  .20578 -.01396
    2000 "173879543"  .20528   .10358  .20407   .0497
    2001 "201912890"  .20651    .0046  .20717  .00467
    1998 "631138735"  .20671   .00035  .20503 -.00052
    2000 "110070106"  .20678   .03369  .20461  .03631
    2002 "21320033X"  .20684   .05416  .21207  .01086
    2002 "726903743"  .20712   .00237  .20823  .05014
    2000 "720172556"  .20803  -.00871  .20149 -.00527
    2000 "254986420"   .2105    .0777  .21277  .01675
    2003 "139371957"  .21101   .00312  .20083  .00256
    2003 "174435346"  .21133  -.03185  .21501  .01187
    2003 "190000692"  .21175   -.0213  .21464  .00931
    1998 "132518725"  .21213   .00316  .21402  .02689
    1998 "216783577"  .21231   .13529  .21838  .04792
    1998 "216577378"  .21248   .00004  .21189  .01197
    1999 "133032428"  .21272   .12988  .21779  .01746
    1999 "155582500"  .21275   .00659  .20405  .02691
    2000 "140710557"  .21377    .0011  .20668 -.01348
    2000 "146341561"  .21382   .02948  .21003  .00727
    1999 "165845626"  .21401   .00324  .21112 -.00407
    2003 "221860813"  .21413   .02531  .20972  .01281
    2001 "101129719"   .2149   .00783   .2146 -.01253
    2000 "15782098X"  .21499   .00682   .2147  .00699
    2000 "134908655"  .21556        0  .21311  .04651
    1999 "280960961"  .21609   .00102  .21997  .06999
    2002 "169358382"  .21673   .02144  .21746  .04879
    1999 "115970162"  .21676   .00702  .21174 -.01479
    2002 "400614546"  .21695  -.09337  .21071  .03537
    2000 "168539878"  .21699   .00265  .22662  .00575
    2002 "728684919"  .21723   .00051  .22409  .01043
    1998 "163567933"  .21777  -.12689  .21634  -.0209
    1999 "103696944"  .21814   .06337  .21119 -.00038
    1999 "114791736"  .21819   .01885  .21629  .02021
    2000 "218033657"  .21849   .00018  .21826 -.00492
    2003 "203133452"  .21933  -.01245   .2229   .0267
    1998 "107770136"  .21935   .08724   .2161   .0004
    2002 "191522695"  .22034   .00413  .21885  .01195
    1999 "705054770"  .22175    .0029  .21056   .0385
    1998 "170954421"  .22281   .14051  .22612 -.01602
    1998 "126914529"  .22296   .01417  .21854 -.00082
    2001 "132271355"  .22317   .01384  .22399 -.01017
    2002 "16363880X"  .22442   .00419  .21071  .03537
    2000 "202380427"  .22482  -.08363  .22595  .00816
    2000 "178858063"  .22523   .01255  .22673 -.01292
    1999 "132820617"  .22532   .00005  .22699  .02111
    2003 "744034481"  .22598   .23336  .22474  .02751
    1998 "110115896"  .22647  -.05761  .22309  .00694
    2001 "216583153"   .2265   .00828  .22272 -.00681
    2003 "170530215"  .22653   .00064  .22519 -.00532
    1999 "27030918X"  .22682    .0051  .23307 -.01278
    2002 "16903410X"  .22683   .01214  .21122 -.01737
    2003 "711591513"   .2274   .02011  .23189 -.01598
    2001 "700799634"  .22843   .03025  .22489 -.00777
    1998 "135896174"  .22896  -.11852   .2315  .00568
    2000 "147930616"  .22994   .00608  .22458  .03387
    2003 "109106749"   .2305  -.02348  .22993 -.00726
    1998 "227764371"  .23078   .03458  .23571  .01054
    1999 "117291470"  .23168   .03485  .23216  .00074
    1999 "171076157"  .23296   .00892  .23719 -.01158
    2003 "101366968"  .23353    .2057  .23026  .01666
    2001 "169789828"  .23414        0  .24083  .01685
    1999 "184281811"  .23465   .00194  .23719 -.01158
    1998 "250817451"  .23513   .01793  .23257  .01677
    2003 "198584098"  .23687   .00937  .23322  .01823
    1998 "167490806"  .23708   .00013  .23369 -.01075
    2000 "147958280"  .23923   .00055  .23435  .00635
    2002 "171077045"  .23935   .00387  .23739  .00747
    2001 "209551110"  .23984   .02884  .24347  .01198
    2001 "228283238"  .24019   .04667  .24347  .01198
    2002 "171070812"   .2413   .01095  .24535  .02306
    2000 "166600997"  .24144   .00288  .23676 -.01216
    1998 "166360700"  .24177   .01947  .24538  .00972
    1998 "118212236"  .24194  -.10463  .23981  .02549
    1999 "164033052"  .24199   .00578  .25108  .03255
    1999 "109551383"  .24238   .00214  .22888  .02818
    1999 "163567490"  .24265   .00054  .24211  .02287
    2003 "221742112"   .2432        0  .23883 -.02296
    2002 "211950225"  .24494  -.09337  .24062 -.00107
    1998 "205402708"  .24728   .00242  .25177 -.00723
    1999 "146718090"  .24806   .00015  .25785 -.00418
    1999 "158321330"  .24854  -.05011  .25258  .00018
    2003 "173351028"     .25   .05087  .24732  .03723
    2000 "140829494"  .25015   .01956  .24367  .01608
    1999 "139711405"  .25044   .01606  .25433  .00238
    1998 "190459831"  .25073   .04427  .24538  .00972
    2001 "101209209"  .25117   .00189  .24843  .00236
    1999 "229850011"  .25137   .00169  .24966  -.0055
    2003 "165011023"  .25155  -.03711  .24732  .03723
    1999 "250038466"  .25244   .00281  .25024  .01376
    2003 "731797949"  .25285   .01352  .24614 -.02121
    1998 "132426775"  .25289   .00019  .25816 -.01346
    2000 "201910801"  .25439   .00536    .261   .0134
    1999 "250852011"  .25443   .00101   .2555  .01355
    1998 "184281803"  .25709   .03627  .26606 -.01757
    2001 "140131053"  .25727   .01709  .25486   .0003
    1998 "132462952"  .25779   .00822  .25477  .00489
    1999 "190339215"  .25869   .00889  .26951  .00364
    2002 "202895885"  .25919   .01875  .25841  .00619
    2003 "10315706X"   .2594    .1617  .24612  .21888
    1998 "143037951"  .25982   .00342  .25931 -.00244
    2002 "10321858X"  .26003  -.01478  .25259  .00012
    1998 "201971970"  .26023   .01391  .25493  .02915
    1999 "17700105X"  .26102    .0002  .26679  .00215
    1998 "218431102"  .26163  -.01051  .26328  .00265
    1999 "108018763"  .26178   .02057  .25652 -.00272
    2001 "703798645"  .26184   .14618  .25917  -.0002
    1998 "187605201"  .26241   .00051  .26328  .00265
    2000 "700900552"  .26352   .01283  .26261  .01831
    2001 "218651586"  .26474   -.0156  .26075  .00494
    2003 "711427641"  .26482   .03133  .26408  .02572
    1999 "200280676"  .26526   .01831  .26191  -.0002
    2002 "140149114"  .26581  -.02784   .2593  .01013
    2003 "187286722"  .26624   .00403  .28661  .02617
    1999 "167313412"  .26646   .01421  .26785 -.01279
    2001 "707526233"  .26833  -.03368  .26373  .01037
    2001 "177720775"  .26879    .0028  .26697   .0064
    1999 "21706059X"  .26907   .05371  .27353   .0105
    2002 "61530174X"  .26919   .14224  .27011  .00926
    1999 "25011051X"  .27001   .00349  .26527  .01427
    2002 "198281783"  .27013  -.02596  .28033  .03004
    2002 "177707263"  .27091   .00022  .26893  .00319
    2001 "222790044"  .27094  -.10077  .26672  .00668
    1998 "174641617"  .27254  -.08027  .27111  .01028
    2002 "217330131"  .27422  -.01361  .27617  .00247
    1998 "16356867X"  .27423   .03632  .27414 -.02052
    2001 "120282057"  .27432   .00277  .27836  .02926
    2003 "749058490"   .2752   .01811  .30263  .00557
    1998 "13440588X"  .27531  -.01559   .2825   .0015
    1998 "150820899"  .27557   .00136  .26769   .0906
    2000 "712829773"  .27681  -.09622  .27413  .00111
    1999 "135994487"  .27699   .15259   .2826  .01091
    1999 "250815237"  .27786   .00804  .28253  .00294
    2002 "145709440"    .278   .00207  .28126  .01064
    2000 "163982694"  .27815   .04637  .28089   .0284
    1999 "104902877"   .2789   .00357  .28122  .01215
    2001 "701778237"  .27943    -.051  .27343  .02959
    1998 "218470216"  .27946  -.06302  .28101  .00371
    2000 "173505891"  .28062   .00106  .27509  .03187
    1999 "227684566"  .28065   .02624  .27884  .00591
    2002 "218690884"  .28094   .01864  .28009  .00956
    2001 "250227419"  .28227  -.10077   .2712   .0054
    1999 "161086511"  .28296   .01322  .28823  .01339
    2000 "203149016"  .28442   .01764  .28792   -.002
    1998 "144078798"  .28663   .02519  .28613  .00419
    2002 "163534568"  .28705   .12805  .29953   .1087
    2003 "143006370"  .28768   .00183  .30138  .02106
    2003 "203566709"  .28929   .00707  .28817  .00718
    1999 "147242733"  .28975   .02288  .28483  .02428
    2003 "727525190"  .29237  -.02233  .28817  .00718
    1999 "620726371"  .29247   .05085  .29234  .02077
    2000 "706893461"  .29312   .03983  .29492 -.00604
    1998 "163146450"  .29426   .00011  .29661 -.01973
    2002 "190502457"  .29444   .01927  .29953   .1087
    1999 "142913059"  .29635  -.10828  .28968 -.00169
    1999 "70576609X"  .29702    .0352  .29227  .00774
    1999 "147380132"  .29758   .00189  .29125 -.00192
    1998 "271752757"  .29783   .02293  .29772 -.01867
    2000 "217226844"  .29851   .01428   .2976 -.00407
    2003 "102286628"  .29957   .03755  .29084  .01728
    1999 "130607070"  .30001   .05837   .2997  .01713
    1998 "224240759"   .3015  -.12689  .30189  .00077
    2001 "120816012"  .30161   .00003  .28775  .00588
    1998 "19046020X"  .30283   .00056  .30062  .05296
    1998 "147935134"  .30434   .04125   .3004   .0009
    1998 "704847832"  .30463   .04536   .3018 -.00989
    1998 "191522142"  .30529  -.02012  .30116  .00084
    2001 "207552136"  .30625   .00061  .30655   .0301
    2002 "727983570"  .30755    .0028  .30431  .03198
    2002 "140334109"  .30959  -.01405  .30802 -.01839
    1998 "126926001"  .31288     .003  .31438 -.00058
    2003 "708607799"  .31341   .00259  .30486  .01155
    2002 "738710949"  .31534    .0075  .31307  .01818
    2000 "142418605"  .31862   .03794  .31791  .00792
    2003 "165006195"  .32104  -.01503  .32114   .0095
    1999 "143038655"  .32144    -.026  .31604 -.00382
    1999 "201913228"  .32145  -.00161  .31976  .00724
    2001 "200931192"  .32164   .07882     .31 -.00066
    2000 "103232015"  .32289   .16439  .32021 -.01443
    2002 "193529946"  .32305   .00006  .32751  .01303
    1999 "139772128"  .32324   .01328  .32082  .00726
    1998 "13314602X"  .32364   .12565  .32074 -.00219
    1999 "22267318X"  .32368   .06007  .32153  .01301
    2003 "731840978"   .3254   .00979  .30059  .00176
    1999 "706625707"  .32589  -.00071  .32429  .01119
    2000 "201968755"   .3261  -.01713  .31301  .01427
    2000 "710924494"  .32679   .06518  .32904 -.00456
    1999 "138056648"  .33114   .00018  .33432 -.01351
    2002 "135897046"  .33125   .00056  .34348  .00343
    2000 "129170991"  .33368   -.0024   .3442 -.00024
    1999 "213550786"  .33381   .02715  .32957  .01303
    2002 "101101628"  .33551   .00612  .31864 -.02373
    1999 "128511002"  .33715   .02773  .33723  .02206
    2003 "182260187"  .33903  -.00696  .33848  .02309
    1999 "147108331"  .33917   .00259   .3384 -.00556
    1998 "126926474"  .33949   -.0724  .33592 -.00377
    1999 "172872729"  .33972   .01536   .3384 -.00556
    2002 "206634340"  .33997  -.09081  .33578  .01736
    2003 "198603930"  .33997    .0004  .34612 -.00637
    1998 "244733454"  .34049   .00316  .33886 -.02318
    1998 "168736919"  .34236  -.12689  .33576   .0034
    2000 "238799721"  .34238  -.00021  .34261  .00447
    2001 "201998356"   .3427   .03266   .3365 -.01545
    1999 "191191996"  .34291   .00103   .3383  .00148
    2000 "209996021"  .34462   .00521  .34961  .02287
    2002 "166732594"  .34548   .01576   .3777  .02212
    2000 "17582225X"  .34585   .00995  .34961  .02287
    1998 "137165295"  .34802   .00285  .34472 -.01063
    2001 "158374678"  .35017   .02087  .34061  .01294
    2002 "101392162"  .35108   .00417  .34717    .022
    2001 "203051502"   .3516    .0001  .34888 -.00012
    2002 "190452242"  .35196   .00895  .33093 -.01148
    2002 "711773595"  .35249   .20697   .3543 -.00186
    1998 "123993807"   .3532  -.02716  .35231 -.00013
    1998 "167571315"  .35412   .03404  .35052 -.00377
    2003 "190457684"  .35427   .05917  .35657 -.00478
    2001 "170070052"  .35721  -.10077  .35497   .0255
    2001 "182260566"  .35873   .00929  .35369  .02516
    end


    this is only part of the data for the size sake, and we could not attach a dat file here?

    thanks in advance for any suggestions.



  • #2
    How large is your data set? Just because the variables match within 10% doesn't mean that the difference will not be statistically significant. If the sample size is large enough, that level of difference can easily be statistically significant. The same would be true for a match within 1%, or .01%, or anything other than zero.

    Let's take a step back. The purpose of checking the closeness of the match is to reassure yourself that the matching will reduce confounding bias in your ultimate analysis. For that purpose, statistical significance of the differences is an irrelevant criterion. I know, people perform these t-tests all the time. But it's based on a misunderstanding and it shouldn't be done.

    In your ultimate analysis comparing the outcome in two groups, the concern is that whatever you find may be influenced by the differences between those groups on your covariates, rather than the group effect itself. Now, what does a difference between the groups on a covariate do? More or less, it contributes to the difference in outcomes by an amount that is the magnitude of the group difference on the covariate multiplied by the marginal effect of the covariate on the outcome. If that contribution is appreciable, relative to the magnitude of the actual group effect, then you have confounding bias. When you match, you reduce the magnitude of the group difference on the covariate. Typically it isn't possible to make it zero altogether, but you can reduce it. If you reduce it so that when that difference gets amplified by the covariate's marginal effect on outcome is small relative to the group effect, then you have satisfactorily reduced the confounding bias.

    So you shouldn't be looking at t-tests. You should look at the magnitudes of the group differences on the covariates. If, taking into account how strongly those covariates are associated with the outcome, those magnitudes are small enough that their contribution to an outcome difference will be negligibly small, you are OK. Otherwise you are not. Statistical significance has nothing to do with it.

    Comment


    • #3
      thanks Clyde, much appreciate for you insightful explanation. I have around 19000 obs after matching(1000 cases, one case may match with multiple controls,). and I think one of the reason why there is significant result is that the co-variate of interest have quite a wide range (from 0 to 3), when match each case with a fixed interval, problems may arise. when I change matching interval into 0.95-1.05 rather than 0.9-1.1, the significance has gone.


      what you said is really reasonable and clear, but I don't know whether I understand you correctly or not , anyway, here it is.

      base on what you said, I may should estimate the marginal effect of the co-variate of interest on the outcome then chose an reasonable and efficient matching interval to reduce the confounding bias.

      and what should I say in the paper that I actually got a nice match without using testing methods like t test.

      Comment


      • #4
        I agree with your interpretations of what I said.

        And, yes, with 19,000 observations, t-tests will be very sensitive to tiny differences that, for purposes of confounding bias control, are ignorable.

        Comment

        Working...
        X