Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Time series regression: Daily predictor and weekly outcome

    Hello all,

    I am using Stata 13.1 on Windows. This is my first time posting to Statalist, so please correct any foibles so that I can learn.

    I am attempting to perform a random effects regression on time series data. My question is: the periodicity of my predictor is daily; whereas the periodicity of my outcome variable is weekly (and in some cases, monthly; for now we can set aside that complication). I know how to merge the two datasets together, but this will create an unbalanced panel, since I will only have data for my outcome variable once a week (or month). Therefore, I am concerned that xtreg will only consider the values of my predictor variables for the end-of-week (or month) day on which I happen to have outcome data, which is not appropriate. How can I fit a model which incorporates the information from my daily predictor to predict my weekly outcome?

    I have a dataset with my predictor variable, which is an index score estimating the intensity of policymaking activity for each day of the COVID pandemic, by each national government, as follows:

    Code:
    * Example generated by -dataex-. To install: ssc install dataex
    clear
    input str32 countrystr str21 modtype str10 date_policy double(med_est high_est low_est sd_est) float(date country iso3numeric) byte MARKER str3 iso3c float iso
    "Afghanistan" "Health Resources" "2020-01-01"   25.9722126250969 32.219066858511304 20.560525573413777 3.6037616464112268 21915 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-02"   25.9722126250969 32.219066858511304 20.560525573413777 3.6037616464112268 21916 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-03"   25.9722126250969 32.219066858511304 20.560525573413777 3.6037616464112268 21917 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-04"   25.9722126250969 32.219066858511304 20.560525573413777 3.6037616464112268 21918 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-05"   25.9722126250969 32.219066858511304 20.560525573413777 3.6037616464112268 21919 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-06" 26.144121945825823 31.377526554406607 21.348419229991727  3.173195354369845 21920 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-07" 26.144121945825823 31.377526554406607 21.348419229991727  3.173195354369845 21921 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-08" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21922 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-09" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21923 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-10" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21924 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-11" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21925 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-12" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21926 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-13" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21927 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-14" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21928 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-15" 25.921784478312304 31.188290700980847 21.581555803792725 2.8679207884199136 21929 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-16" 25.918555992625684  30.90023253851845  21.76324567450482  2.809923064610894 21930 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-17" 25.930770616910387 30.644527200029547 21.708434657350082 2.7606465152938195 21931 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-18" 25.930770616910387 30.644527200029547 21.708434657350082 2.7606465152938195 21932 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-19" 25.930770616910387 30.644527200029547 21.708434657350082 2.7606465152938195 21933 1 4 1 "AFG" 1
    "Afghanistan" "Health Resources" "2020-01-20" 25.895474535667418 30.630445381222174  21.60560935367298 2.7430899735147274 21934 1 4 1 "AFG" 1
    end
    format %td date
    label values country country
    label def country 1 "Afghanistan", modify
    label values iso iso
    label def iso 1 "AFG", modify
    So for this dataset, I would
    Code:
     xtset country date
    However, my outcome variable is excess mortality p-scores, which has a weekly periodicity for most countries (monthly for others, but we can set aside that complication for now; if I need to, I will analyze weekly vs monthly groups of countries separately). It looks like this:

    Code:
    * Example generated by -dataex-. To install: ssc install dataex
    clear
    input float(p_scores_all_ages enddate month week timeunit country) str3 iso3c
      .47 21919 . 3120 2 6 "AUS"
     1.33 21926 . 3121 2 6 "AUS"
     2.05 21933 . 3122 2 6 "AUS"
     7.06 21940 . 3123 2 6 "AUS"
     2.91 21947 . 3124 2 6 "AUS"
     3.71 21954 . 3125 2 6 "AUS"
     6.68 21961 . 3126 2 6 "AUS"
      5.7 21968 . 3127 2 6 "AUS"
     8.37 21975 . 3128 2 6 "AUS"
     3.54 21982 . 3129 2 6 "AUS"
     4.89 21989 . 3130 2 6 "AUS"
     7.64 21996 . 3131 2 6 "AUS"
    10.62 22003 . 3132 2 6 "AUS"
    13.51 22010 . 3133 2 6 "AUS"
     8.79 22017 . 3134 2 6 "AUS"
     4.02 22024 . 3135 2 6 "AUS"
     5.84 22031 . 3136 2 6 "AUS"
      2.3 22038 . 3137 2 6 "AUS"
     3.51 22045 . 3138 2 6 "AUS"
     -.88 22052 . 3139 2 6 "AUS"
    end
    format %td enddate
    format %tm month
    format %tw week
    label values timeunit timeunit
    label def timeunit 2 "weekly", modify
    label values country country
    label def country 6 "Australia", modify
    For this dataset alone, I would
    Code:
     xtset country enddate, delta (7 days)
    Unfortunately, I did not create the daily indices I am using as a predictor, so I cannot remodel them as a weekly index to match my outcome variable. They are from Kubinec R, Barcel J, Goldszmidt R, et al. Statistically Validated Indices for COVID-19 Public Health Policies. SocArXiv. 2021;1–29.

    I would appreciate any ideas. Perhaps I am misunderstanding how xtreg works, as I am new to panel data analysis.

    Many thanks,
    Sarah






  • #2
    Well, before you worry about specific code, I think you have to make some decisions about how you want to conceptually associate the weekly/monthly data to the daily data. There are a few ways to go with this. One possibility is to -collapse- the daily data to weekly means (or medians, or something like that) and then use the week as the time unit for your analysis. I sense, though you do not say it in so many words, that you don't want to do that.

    If you don't want to do that, then you have to do something a little fancier than simple merging so that each weekly/observation gets matched to each date of that week in the corresponding data. But which dates correspond go which week? In your weekly data example, all of the dates in the variable enddate are the Sunday of the Stata-week that is found in the variable week. Is that always the case in the real (weekly) data? Do you want to mate these with the 7 days that end with that date (as the name enddate suggests)? Or is it more sensible in terms of the substance of your research questions to mate these with the 7 days starting with that date, or perhaps some 7 day window which includes that date (e.g. from the Wednesday preceding through the Tuesday following)?

    When you post back, please provide new example data, this time using the same country in both examples. Also, since the monthly data will have to be handled differently, please show some examples of that, and in the daily data include some that could potentially match up with those. It's important to have example data that can actually test the code. For code involving combining data sets, that means that the key variables in the example data sets must take on values such that some of the observations can match with each other.

    Comment


    • #3
      Dr. Schechter, Thank you for your questions and suggestions thus far.

      You are correct that I don't think it would be valid to -collapse- the daily predictor variable into weekly or monthly means/medians etc. This variable is a constructed index of the intensity of policymaking activity that was created using a time-varying ideal point model (see Kubinec's article quoted above, or Kubinec R. Generalized Ideal Point Models for Time-Varying and Missing-Data Inference. 2019;1–46 for more details). The time-varying portion varies daily (each day is estimated taking into account the information from the previous day).

      For the outcomes, the name enddate is informative as you guessed. The variable is a summary statistic of the number of excess deaths that occurred during the previous ISO 8601 week (or calendar month), so logically matches up with the week- or month-long period that precedes it.

      For the data examples: I have included more observations for the predictors than for the outcomes, so that there are a couple of weeks/months to match on respectively -- apologies in advance that the data examples are thus a bit long. String dates included to make this intelligible outside of Stata context.

      Here is the predictor for a weekly-outcome country, Australia:

      Code:
      * Example generated by -dataex-. To install: ssc install dataex
      clear
      input float country str3 iso3c float iso str10 date_policy float date double(med_est high_est low_est sd_est)
      9 "AUS" 9 "2020-01-01" 21915  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
      9 "AUS" 9 "2020-01-02" 21916  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
      9 "AUS" 9 "2020-01-03" 21917  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
      9 "AUS" 9 "2020-01-04" 21918  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
      9 "AUS" 9 "2020-01-05" 21919  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
      9 "AUS" 9 "2020-01-06" 21920 25.883226019166663 31.208711177979144  21.07105076038048 3.0316880398138113
      9 "AUS" 9 "2020-01-07" 21921 25.883226019166663 31.208711177979144  21.07105076038048 3.0316880398138113
      9 "AUS" 9 "2020-01-08" 21922 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-09" 21923 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-10" 21924 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-11" 21925 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-12" 21926 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-13" 21927 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-14" 21928 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-15" 21929 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
      9 "AUS" 9 "2020-01-16" 21930 25.851743415493406  30.51704326964375 21.416894198954054  2.699060041005349
      9 "AUS" 9 "2020-01-17" 21931 25.866859356450394   30.3964082086573 21.656454179705058 2.6812171339593323
      9 "AUS" 9 "2020-01-18" 21932 25.866859356450394   30.3964082086573 21.656454179705058 2.6812171339593323
      9 "AUS" 9 "2020-01-19" 21933 25.866859356450394   30.3964082086573 21.656454179705058 2.6812171339593323
      9 "AUS" 9 "2020-01-20" 21934 25.948763684517793  30.70695896430282  21.79084046455128 2.6810102805133575
      9 "AUS" 9 "2020-01-21" 21935 25.820615726511136 30.389318769511192  21.68765975398683 2.7399689123561126
      9 "AUS" 9 "2020-01-22" 21936  25.83255700878032 30.425159869453513 21.720449568338942 2.6805416641913946
      9 "AUS" 9 "2020-01-23" 21937  25.74412380514692 30.588516890977697  21.66102710380498 2.7604549693391722
      9 "AUS" 9 "2020-01-24" 21938  25.80148271926748  30.51417942767968 21.560363473380583  2.764124005985674
      9 "AUS" 9 "2020-01-25" 21939 25.794417621359237 30.417447418499382  21.69565111260071  2.662477223358356
      9 "AUS" 9 "2020-01-26" 21940 25.765576171302026 30.361184851380276  21.82987772626589 2.6222657281969566
      9 "AUS" 9 "2020-01-27" 21941  25.77845616219077  30.50171032911958 21.747328525854243 2.6397231340101475
      9 "AUS" 9 "2020-01-28" 21942 25.835318625271412 30.268493550631568  21.78675639126671  2.612931456119607
      9 "AUS" 9 "2020-01-29" 21943 25.909210177806926  30.46716844642483 21.916852129170223  2.601202772041947
      9 "AUS" 9 "2020-01-30" 21944 25.957113311595627  30.32782053186286  21.86316520225227  2.550073774485608
      end
      format %td date
      label values country country
      label def country 9 "Australia", modify
      label values iso iso
      label def iso 9 "AUS", modify
      Here is the outcome for Australia:

      Code:
      * Example generated by -dataex-. To install: ssc install dataex
      clear
      input str10 date float(enddate month week timeunit country) str3 iso3c float p_scores_all_ages
      "2020-01-05" 21919 . 3120 2 6 "AUS"  .47
      "2020-01-12" 21926 . 3121 2 6 "AUS" 1.33
      "2020-01-19" 21933 . 3122 2 6 "AUS" 2.05
      "2020-01-26" 21940 . 3123 2 6 "AUS" 7.06
      "2020-02-02" 21947 . 3124 2 6 "AUS" 2.91
      "2020-02-09" 21954 . 3125 2 6 "AUS" 3.71
      end
      format %td enddate
      format %tm month
      format %tw week
      label values timeunit timeunit
      label def timeunit 2 "weekly", modify
      label values country country
      label def country 6 "Australia", modify

      Now here is the predictor for a monthly-outcome country, Japan:

      Code:
      * Example generated by -dataex-. To install: ssc install dataex
      clear
      input float country str3 iso3c float iso str10 date_policy float date double(med_est high_est low_est sd_est)
      84 "JPN" 84 "2020-01-01" 21915 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
      84 "JPN" 84 "2020-01-02" 21916 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
      84 "JPN" 84 "2020-01-03" 21917 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
      84 "JPN" 84 "2020-01-04" 21918 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
      84 "JPN" 84 "2020-01-05" 21919 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
      84 "JPN" 84 "2020-01-06" 21920 28.055412672506485  33.42524217513225  22.81421530625869  3.210519755735504
      84 "JPN" 84 "2020-01-07" 21921 28.055412672506485  33.42524217513225  22.81421530625869  3.210519755735504
      84 "JPN" 84 "2020-01-08" 21922 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-09" 21923 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-10" 21924 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-11" 21925 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-12" 21926 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-13" 21927 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-14" 21928 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-15" 21929 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
      84 "JPN" 84 "2020-01-16" 21930 28.102445526289166 33.444160689651646  23.34182218112627  2.984003448253295
      84 "JPN" 84 "2020-01-17" 21931 28.754705328768807  33.77371934851828 24.093067855296994 2.8727637515942113
      84 "JPN" 84 "2020-01-18" 21932 28.754705328768807  33.77371934851828 24.093067855296994 2.8727637515942113
      84 "JPN" 84 "2020-01-19" 21933 28.754705328768807  33.77371934851828 24.093067855296994 2.8727637515942113
      84 "JPN" 84 "2020-01-20" 21934  29.07509365391697  34.54862559511027  24.66396549177164 3.0166374155464726
      84 "JPN" 84 "2020-01-21" 21935 29.621974596682865 34.471948135614774 24.948525497283846 2.9680508014743614
      84 "JPN" 84 "2020-01-22" 21936 29.924524958423405   35.1675192484036  25.38818104467067 3.0021694908521632
      84 "JPN" 84 "2020-01-23" 21937 30.423480229046348  35.19125175369077 25.393226218673938 3.0486100487446643
      84 "JPN" 84 "2020-01-24" 21938 30.535635833314075  35.53664714000712 25.425549511224222 3.0933253566292946
      84 "JPN" 84 "2020-01-25" 21939  30.92069503912313  36.15503275593188 26.401995647908556 3.0296502118490145
      84 "JPN" 84 "2020-01-26" 21940 31.638482736689767  37.35524882397148 26.309486654074497 3.2596085611754653
      84 "JPN" 84 "2020-01-27" 21941  32.69791633451514  38.36378474720795 27.449443620244033 3.2754921811648177
      84 "JPN" 84 "2020-01-28" 21942 34.125925625787474 39.521237557244476  28.81621494604594  3.285679494915828
      84 "JPN" 84 "2020-01-29" 21943 36.049661661933854 42.045703483409206  31.12689771980661 3.3835851496477023
      84 "JPN" 84 "2020-01-30" 21944  39.57308893360567  45.64250404105175  34.01904287969073  3.459546989884957
      84 "JPN" 84 "2020-01-31" 21945 44.373270639867364 50.154733102524396  38.68423876922484  3.504511980840289
      84 "JPN" 84 "2020-02-01" 21946   50.0622371089655   56.0823758737659  43.85692720277268 3.7332746456202055
      84 "JPN" 84 "2020-02-02" 21947  54.40538483064739 60.416748565993935 48.587713139874914 3.6275210408925784
      84 "JPN" 84 "2020-02-03" 21948 58.231315457290684  63.63197439771619  51.94406982591783 3.5022513402650124
      84 "JPN" 84 "2020-02-04" 21949  60.93472154205432  66.73325800599984 54.956098835917444  3.526639422565003
      84 "JPN" 84 "2020-02-05" 21950  63.42032521819489  68.77063829024183  58.05005156732581  3.294628993740025
      84 "JPN" 84 "2020-02-06" 21951  66.09938254275909  71.19137586163578  60.77710511132896 3.2393985968202643
      84 "JPN" 84 "2020-02-07" 21952  67.78898211749454  72.70273517346203  62.28318632257358 3.1362889377847316
      84 "JPN" 84 "2020-02-08" 21953  68.25844460261769  73.06406546736031   63.0930746571133 3.0518636801583106
      84 "JPN" 84 "2020-02-09" 21954  68.66230642055318  73.63573955298871  63.73100790165183 3.0689273309186174
      84 "JPN" 84 "2020-02-10" 21955  69.27038944920021  74.04628195869506 63.939744672016076 3.0555606401784368
      84 "JPN" 84 "2020-02-11" 21956  69.98387725648294  74.83302259749509  64.99484012072975   3.06851549923668
      84 "JPN" 84 "2020-02-12" 21957  71.11226580902591  75.74042605559716  65.79861827081015  3.027445886878842
      84 "JPN" 84 "2020-02-13" 21958  72.38121418364418  77.24848677227004  67.51366702035507 2.9406341653476775
      84 "JPN" 84 "2020-02-14" 21959  73.88576744724963   78.3817177347113  69.23423968979613  2.788114185746373
      84 "JPN" 84 "2020-02-15" 21960  75.00775938114569  79.23420723016264  70.25563475916177  2.800576410604241
      84 "JPN" 84 "2020-02-16" 21961  75.64270798648118  79.81592166610562  71.22070357794001  2.624923820702171
      84 "JPN" 84 "2020-02-17" 21962  76.33524928351056  80.21260610408322  71.87783979351474 2.5754268247530936
      84 "JPN" 84 "2020-02-18" 21963  77.11439572172455  80.99266974025478  72.49350456968041 2.5107066646258036
      84 "JPN" 84 "2020-02-19" 21964  77.83559985404683   81.6164568492656  73.53319241516445 2.4373487189680922
      84 "JPN" 84 "2020-02-20" 21965   78.8633839402253  82.64850283016703  74.76057540816623 2.3877292884558314
      84 "JPN" 84 "2020-02-21" 21966  79.74789955826753  83.03537132007847  75.59977386334643 2.2953466582793056
      84 "JPN" 84 "2020-02-22" 21967  80.50290831334837  84.00035495857415  76.36761108008017 2.3095569992601748
      84 "JPN" 84 "2020-02-23" 21968  81.48104722273156  84.77623046305328  77.41921616057617 2.1758714602996014
      84 "JPN" 84 "2020-02-24" 21969  82.47891181940943  85.62476458780847  78.67876487160672 2.0933587137072878
      84 "JPN" 84 "2020-02-25" 21970  84.00323132020631  86.71770083249646  80.50303810872403 1.9091415669856315
      84 "JPN" 84 "2020-02-26" 21971  84.75122737888177  87.40252308331957  81.44625594209639 1.7864629506615648
      84 "JPN" 84 "2020-02-27" 21972  85.40764358295624   88.2281649801761  82.35155009510521  1.830133464078396
      84 "JPN" 84 "2020-02-28" 21973  85.73262462145902  88.55535681095921  82.54683810254186 1.8205054450709437
      84 "JPN" 84 "2020-02-29" 21974  85.98558170046167  88.73411112054548  82.82612450246688 1.7809275333650822
      84 "JPN" 84 "2020-03-01" 21975  86.29433483513816   88.9561317441924  83.05214206869577  1.795590476885897
      84 "JPN" 84 "2020-03-02" 21976  86.35127115475481  88.92544230621652  83.14598318120645 1.7588196100296627
      end
      format %td date
      label values country country
      label def country 84 "Japan", modify
      label values iso iso
      label def iso 84 "JPN", modify
      And here is the outcome for Japan:

      Code:
      * Example generated by -dataex-. To install: ssc install dataex
      clear
      input str10 date float(enddate month week timeunit country) str3 iso3c float p_scores_all_ages
      "2020-01-31" 21945 720 . 1 48 "JPN" -1.19
      "2020-02-29" 21974 721 . 1 48 "JPN"   .65
      "2020-03-31" 22005 722 . 1 48 "JPN"  1.18
      "2020-04-30" 22035 723 . 1 48 "JPN"   4.9
      "2020-05-31" 22066 724 . 1 48 "JPN"  2.53
      "2020-06-30" 22096 725 . 1 48 "JPN"  2.76
      end
      format %td enddate
      format %tm month
      format %tw week
      label values timeunit timeunit
      label def timeunit 1 "monthly", modify
      label values country country
      label def country 48 "Japan", modify
      Please let me know of any ideas you have. As you identified above, the merge is less the issue than the conceptual relation between these data in time when I model them. Thank you again for your help.

      Sarah

      Comment


      • #4
        OK. In the code below, I have eliminated the string version of dates that you included because they are of no value computationally, and because of the way they are named, they mess up the code! The first step is to split your predictor and outcome data sets into the countries that are monthly reporters and those that are weekly reporters, just as you did for your example data. They have to be processed separately, but the end results can be appended together to give you a full data set again.

        The monthly countries are fairly straightforward:
        Code:
        * Example generated by -dataex-. For more info, type help dataex
        clear
        input float country str3 iso3c float(iso date) double(med_est high_est low_est sd_est)
        84 "JPN" 84 21915 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
        84 "JPN" 84 21916 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
        84 "JPN" 84 21917 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
        84 "JPN" 84 21918 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
        84 "JPN" 84 21919 27.540479746053776 33.897559685498216  22.08276317020514 3.5891905005152713
        84 "JPN" 84 21920 28.055412672506485  33.42524217513225  22.81421530625869  3.210519755735504
        84 "JPN" 84 21921 28.055412672506485  33.42524217513225  22.81421530625869  3.210519755735504
        84 "JPN" 84 21922 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21923 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21924 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21925 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21926 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21927 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21928 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21929 27.921566068738645  33.05544837732028 23.098496911273173 3.0102640824229527
        84 "JPN" 84 21930 28.102445526289166 33.444160689651646  23.34182218112627  2.984003448253295
        84 "JPN" 84 21931 28.754705328768807  33.77371934851828 24.093067855296994 2.8727637515942113
        84 "JPN" 84 21932 28.754705328768807  33.77371934851828 24.093067855296994 2.8727637515942113
        84 "JPN" 84 21933 28.754705328768807  33.77371934851828 24.093067855296994 2.8727637515942113
        84 "JPN" 84 21934  29.07509365391697  34.54862559511027  24.66396549177164 3.0166374155464726
        84 "JPN" 84 21935 29.621974596682865 34.471948135614774 24.948525497283846 2.9680508014743614
        84 "JPN" 84 21936 29.924524958423405   35.1675192484036  25.38818104467067 3.0021694908521632
        84 "JPN" 84 21937 30.423480229046348  35.19125175369077 25.393226218673938 3.0486100487446643
        84 "JPN" 84 21938 30.535635833314075  35.53664714000712 25.425549511224222 3.0933253566292946
        84 "JPN" 84 21939  30.92069503912313  36.15503275593188 26.401995647908556 3.0296502118490145
        84 "JPN" 84 21940 31.638482736689767  37.35524882397148 26.309486654074497 3.2596085611754653
        84 "JPN" 84 21941  32.69791633451514  38.36378474720795 27.449443620244033 3.2754921811648177
        84 "JPN" 84 21942 34.125925625787474 39.521237557244476  28.81621494604594  3.285679494915828
        84 "JPN" 84 21943 36.049661661933854 42.045703483409206  31.12689771980661 3.3835851496477023
        84 "JPN" 84 21944  39.57308893360567  45.64250404105175  34.01904287969073  3.459546989884957
        84 "JPN" 84 21945 44.373270639867364 50.154733102524396  38.68423876922484  3.504511980840289
        84 "JPN" 84 21946   50.0622371089655   56.0823758737659  43.85692720277268 3.7332746456202055
        84 "JPN" 84 21947  54.40538483064739 60.416748565993935 48.587713139874914 3.6275210408925784
        84 "JPN" 84 21948 58.231315457290684  63.63197439771619  51.94406982591783 3.5022513402650124
        84 "JPN" 84 21949  60.93472154205432  66.73325800599984 54.956098835917444  3.526639422565003
        84 "JPN" 84 21950  63.42032521819489  68.77063829024183  58.05005156732581  3.294628993740025
        84 "JPN" 84 21951  66.09938254275909  71.19137586163578  60.77710511132896 3.2393985968202643
        84 "JPN" 84 21952  67.78898211749454  72.70273517346203  62.28318632257358 3.1362889377847316
        84 "JPN" 84 21953  68.25844460261769  73.06406546736031   63.0930746571133 3.0518636801583106
        84 "JPN" 84 21954  68.66230642055318  73.63573955298871  63.73100790165183 3.0689273309186174
        84 "JPN" 84 21955  69.27038944920021  74.04628195869506 63.939744672016076 3.0555606401784368
        84 "JPN" 84 21956  69.98387725648294  74.83302259749509  64.99484012072975   3.06851549923668
        84 "JPN" 84 21957  71.11226580902591  75.74042605559715  65.79861827081015  3.027445886878842
        84 "JPN" 84 21958  72.38121418364418  77.24848677227004  67.51366702035507 2.9406341653476775
        84 "JPN" 84 21959  73.88576744724963   78.3817177347113  69.23423968979613  2.788114185746373
        84 "JPN" 84 21960  75.00775938114569  79.23420723016264  70.25563475916177  2.800576410604241
        84 "JPN" 84 21961  75.64270798648118  79.81592166610562  71.22070357794001  2.624923820702171
        84 "JPN" 84 21962  76.33524928351056  80.21260610408322  71.87783979351474 2.5754268247530936
        84 "JPN" 84 21963  77.11439572172455  80.99266974025478  72.49350456968041 2.5107066646258036
        84 "JPN" 84 21964  77.83559985404683   81.6164568492656  73.53319241516445 2.4373487189680922
        84 "JPN" 84 21965   78.8633839402253  82.64850283016703  74.76057540816623 2.3877292884558314
        84 "JPN" 84 21966  79.74789955826753  83.03537132007847  75.59977386334643 2.2953466582793056
        84 "JPN" 84 21967  80.50290831334837  84.00035495857415  76.36761108008017 2.3095569992601748
        84 "JPN" 84 21968  81.48104722273156  84.77623046305328  77.41921616057617 2.1758714602996014
        84 "JPN" 84 21969  82.47891181940943  85.62476458780847  78.67876487160672 2.0933587137072878
        84 "JPN" 84 21970  84.00323132020631  86.71770083249646  80.50303810872403 1.9091415669856315
        84 "JPN" 84 21971  84.75122737888177  87.40252308331956  81.44625594209639 1.7864629506615648
        84 "JPN" 84 21972  85.40764358295624   88.2281649801761  82.35155009510521  1.830133464078396
        84 "JPN" 84 21973  85.73262462145902  88.55535681095921  82.54683810254186 1.8205054450709437
        84 "JPN" 84 21974  85.98558170046167  88.73411112054548  82.82612450246688 1.7809275333650822
        84 "JPN" 84 21975  86.29433483513816   88.9561317441924  83.05214206869577  1.795590476885897
        84 "JPN" 84 21976  86.35127115475481  88.92544230621652  83.14598318120645 1.7588196100296627
        end
        format %td date
        label values country country
        label def country 84 "Japan", modify
        label values iso iso
        label def iso 84 "JPN", modify
        tempfile predictors
        save `predictors'
        
        * Example generated by -dataex-. For more info, type help dataex
        clear
        input float(enddate month week timeunit country) str3 iso3c float p_scores_all_ages
        21945 720 . 1 48 "JPN" -1.19
        21974 721 . 1 48 "JPN"   .65
        22005 722 . 1 48 "JPN"  1.18
        22035 723 . 1 48 "JPN"   4.9
        22066 724 . 1 48 "JPN"  2.53
        22096 725 . 1 48 "JPN"  2.76
        end
        format %td enddate
        format %tm month
        format %tw week
        label values timeunit timeunit
        label def timeunit 1 "monthly", modify
        label values country country
        label def country 48 "Japan", modify
        tempfile outcomes
        save `outcomes'
        
        use `predictors', clear
        gen month = mofd(date)
        
        merge m:1 iso3c month using `outcomes', keep(match) nogenerate
        The weekly countries are a bit more complicated. The complication arises because Stata does not define weeks in the way you are using the term. In Stata, Jan 1 always marks the start of a new week, and subsequent weeks are defined as beginning every 7 days thereafter (with a fudge for the final week which will contain an extra day, or two in leap years). This is quite different from defining a week as a group 7 days ending on a Sunday. So we can't use Stata's weekly date functions for this match. We have to "roll our own." The following creates a variable called match_week. It is defined so that match_week 0 is the week that begins on Monday, 6 Jan 2020, stored in local macro base_date, and ends on Sunday 12 Jan 2020. (The choice of 6 Jan 2020 as the base date was arbitrary--the only thing that really matters is that it be a Monday.) Then the match_week of a given daily date is defined as the number of days elapsed from that base date divided by 7 (with the remainder discarded). Then we can merge the data on country and match_week and you're good to go.

        Code:
        * Example generated by -dataex-. For more info, type help dataex
        clear
        input float country str3 iso3c float(iso date) double(med_est high_est low_est sd_est)
        9 "AUS" 9 21915  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
        9 "AUS" 9 21916  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
        9 "AUS" 9 21917  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
        9 "AUS" 9 21918  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
        9 "AUS" 9 21919  25.89569216043771 31.723459845454254 20.723735831320525 3.3303311839797796
        9 "AUS" 9 21920 25.883226019166663 31.208711177979144  21.07105076038048 3.0316880398138113
        9 "AUS" 9 21921 25.883226019166663 31.208711177979144  21.07105076038048 3.0316880398138113
        9 "AUS" 9 21922 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21923 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21924 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21925 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21926 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21927 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21928 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21929 25.788773839076075 30.966778184987795 21.537286558855463 2.8211651725435503
        9 "AUS" 9 21930 25.851743415493406  30.51704326964375 21.416894198954054  2.699060041005349
        9 "AUS" 9 21931 25.866859356450394   30.3964082086573 21.656454179705058 2.6812171339593323
        9 "AUS" 9 21932 25.866859356450394   30.3964082086573 21.656454179705058 2.6812171339593323
        9 "AUS" 9 21933 25.866859356450394   30.3964082086573 21.656454179705058 2.6812171339593323
        9 "AUS" 9 21934 25.948763684517793  30.70695896430282  21.79084046455128 2.6810102805133575
        9 "AUS" 9 21935 25.820615726511136 30.389318769511192  21.68765975398683 2.7399689123561126
        9 "AUS" 9 21936  25.83255700878032 30.425159869453513 21.720449568338942 2.6805416641913946
        9 "AUS" 9 21937  25.74412380514692 30.588516890977697  21.66102710380498 2.7604549693391722
        9 "AUS" 9 21938  25.80148271926748  30.51417942767968 21.560363473380583  2.764124005985674
        9 "AUS" 9 21939 25.794417621359237 30.417447418499382  21.69565111260071  2.662477223358356
        9 "AUS" 9 21940 25.765576171302026 30.361184851380276  21.82987772626589 2.6222657281969566
        9 "AUS" 9 21941  25.77845616219077  30.50171032911958 21.747328525854243 2.6397231340101475
        9 "AUS" 9 21942 25.835318625271412 30.268493550631568  21.78675639126671  2.612931456119607
        9 "AUS" 9 21943 25.909210177806926  30.46716844642483 21.916852129170223  2.601202772041947
        9 "AUS" 9 21944 25.957113311595627  30.32782053186286  21.86316520225227  2.550073774485608
        end
        format %td date
        label values country country
        label def country 9 "Australia", modify
        label values iso iso
        label def iso 9 "AUS", modify
        tempfile predictors
        save `predictors'
        
        * Example generated by -dataex-. For more info, type help dataex
        clear
        input float(enddate month week timeunit country) str3 iso3c float p_scores_all_ages
        21919 . 3120 2 6 "AUS"  .47
        21926 . 3121 2 6 "AUS" 1.33
        21933 . 3122 2 6 "AUS" 2.05
        21940 . 3123 2 6 "AUS" 7.06
        21947 . 3124 2 6 "AUS" 2.91
        21954 . 3125 2 6 "AUS" 3.71
        end
        format %td enddate
        format %tm month
        format %tw week
        label values timeunit timeunit
        label def timeunit 2 "weekly", modify
        label values country country
        label def country 6 "Australia", modify
        tempfile outcomes
        save `outcomes'
        
        local base_date = td(6jan2020)
        assert dow(`base_date') == 1 // BASE DATE MUST BE A MONDAY
        
        use `outcomes', clear
        gen match_week = floor((enddate-`base_date'))
        tempfile matchable_outcomes
        save `matchable_outcomes'
        
        use `predictors', clear
        gen match_week = floor((date-`base_date')/7)
        merge m:1 iso3c match_week using `matchable_outcomes', keep(match) nogenerate

        Comment


        • #5
          Dr. Schechter--
          Circling back to this (have had a lot of clinical time recently). Thanks very much for your help. This works perfectly for the merge, and after looking a bit more closely at my predictor variable, I think I am able to summarize it quite well with weekly/monthly means, although I'll continue to test this. The only substantive change I made from your approach was when constructing match_week for the outcome variables, I did also divide by 7 as you did for the predictors (since it seems like Stata is subtracting days, not weeks here).
          Again, thanks for your help!
          Sarah

          Comment


          • #6
            I did also divide by 7 as you did for the predictors (since it seems like Stata is subtracting days, not weeks here).
            Yes, you are quite correct! Thank you for spotting that error and posting your correction.

            Comment

            Working...
            X