Generalized difference in difference

[Tammie Jones]

I'm studying the causal effects of a state policy, nurse practitioner scope of practice law, on various outcomes (3 different studies). States' policies are categorized as restricted, reduced, and full practice. I've simplified to look at just restricted states and states that change policy from restricted to reduced as I'm learning to apply this method for my dissertation. Policy changes at different times for multiple states during the study period (2010-2017). I'm trying to verify whether my data setup is solid.

In this example, my outcome is totnp (count of nurse practitioners). The unit of observation is county/year. The scope of practice policy changes occur at the state level. Independent variables are year (2010-2017), totclinics (total clinics), totpop (total population), unemployment (rate), and fips (group variable state/county fips). time0_rest_red is the time indicator variable where 0 = not treated, 1 = treatment year and beyond.

Given this outcome is a count variable I've elected to use poisson and use xtset to establish a panel dataset. The treatment group is made up from counties in states that change state scope of practice law from restricted to reduced. The control group is made up from counties in states that have restricted scope of practice laws during the entire study period (2010-2017). I have a second potential control group, counties in states that have full practice scope of practice laws during the entire study period; I don't find examples of how to set this up.

xtset fips year
xtpoisson totnp time0_rest_red i.fips i.year if study1a_sample==1, fe vce(robust)

Thanks in advance for weighing in. My sample code is below.

* Example generated by -dataex-. To install: ssc install dataex
clear
input int totnp float time0_rest_red int(year totclinics) long totpop float unemployment long fips float study1a_sample
 18 0 2010  7  82291 11.6 61 1
  2 0 2010  2  13228 16.3 18 1
 29 0 2010  3 119490  9.6 52 1
  0 0 2010  1  11539 12.8 19 1
  2 0 2010  3  23068 17.2 50 1
 22 0 2010  4  67023 10.7 64 1
  8 0 2010  1  21027 12.3 46 1
  2 0 2010  0  17302  9.4 34 1
  2 0 2010  1  14972  8.3 15 1
 18 0 2010  2  41616 12.9 62 1
  7 0 2010  1  83593  9.1 58 1
  5 0 2010  1  43820 17.4 24 1
  6 0 2010  3  34339   11 40 1
  0 0 2010  2   9045 17.3 32 1
 22 0 2010  1  82782  8.2 42 1
341 0 2010  8 658466  9.4 37 1
 33 0 2010  2  80406  8.7 22 1
 52 0 2010  0 101547  8.4 35 1
 24 0 2010  1 195085  6.9 59 1
  4 0 2010  3  20947 12.3  7 1
139 0 2010  3 334811  7.4 45 1
  2 0 2010  2  52947 10.3 57 1
 18 0 2010  1  21452 12.8 44 1
 31 0 2010  3 182265  8.9  2 1
103 0 2010  4 229363  9.2 51 1
  2 0 2010  4  27457 11.7  3 1
 12 0 2010  1  32899  8.2 55 1
  2 0 2010  1  34215 13.7  9 1
 39 0 2010  3  92709  8.8 39 1
  4 0 2010  3  13763 14.5 60 1
 31 0 2010  2 118572  9.7  8 1
 16 0 2010  3  37765  9.1 20 1
  3 0 2010  1  26790  8.9 31 1
 12 0 2010  6  22913 12.6 56 1
  6 0 2010  2  13906  8.7 21 1
 18 0 2010  5  71109 11.6 25 1
  2 0 2010  4  11670 21.3 66 1
  4 0 2010  2  13859 11.3 12 1
 13 0 2010  1  50251  8.5 23 1
  3 0 2010  0  13932 13.8 14 1
  6 0 2010  5  25833 15.6 13 1
  0 0 2010  0  43643  9.2 11 1
 10 0 2010  1  79303  8.5 26 1
 14 0 2010 10  53227  9.7 36 1
  8 0 2010  1  30776 13.3 47 1
 32 0 2010  1  54428  9.8 17 1
  2 0 2010  2  19746 11.4 54 1
  3 0 2010  8  38319 11.1 27 1
  1 0 2010  1  25989  9.3 10 1
  2 0 2010  2  11299 16.3 43 1
  7 0 2010  4  31704 10.2 30 1
  2 0 2010  2  14564 12.9 38 1
  3 0 2010  1  54571  7.8  1 1
 28 0 2010  0 140247  8.1 41 1
  3 0 2010  2  17581 13.3 65 1
 33 0 2010  9 104430  9.5 28 1
135 0 2010 15 412992 10.2 49 1
  7 0 2010  0  49948  7.4 16 1
 13 0 2010  6  93019  8.6 48 1
  4 0 2010  2  15760 13.4 33 1
  7 0 2010  3  22915 10.3  4 1
 86 0 2010  3 194656  8.5 63 1
  2 0 2010  3  10591 16.1 53 1
  4 0 2010  0  17241 11.7 29 1
  6 0 2010  3  24484 15.1 67 1
  3 0 2010  2  10914 14.7  6 1
  3 0 2010  3  57322    9  5 1
 16 0 2011 10  53291  8.7 36 1
 37 0 2011  1  54512    9 17 1
  8 0 2011  0  50526  7.1 16 1
  5 0 2011  8  38095 10.5 27 1
  5 0 2011  2  55267    8  1 1
  3 0 2011  4  11482 19.3 66 1
  2 0 2011  1  14835  8.3 15 1
  3 0 2011  2  17344   14 65 1
  2 0 2011  3  11147 16.6 43 1
  4 0 2011  3  10542 15.3  6 1
  7 0 2011  3  22766  9.9  4 1
  2 0 2011  1  17412  8.4 34 1
 32 0 2011  2 119953  8.7 52 1
 10 0 2011  2  80162  8.3 26 1
 39 0 2011 10 104303  9.1 28 1
 12 0 2011  1  84398  8.3 58 1
 35 0 2011  2  80536  8.1 22 1
 13 0 2011  1  32915  7.8 55 1
  5 0 2011  0  17182 10.4 29 1
  6 0 2011  2  13911  8.7 21 1
  2 0 2011  3  13478 14.5 60 1
  6 0 2011  2  43332 16.2 24 1
  2 0 2011  2  19349 10.7 54 1
126 0 2011  5 232032  9.1 51 1
 99 0 2011  3 197211  8.2 63 1
  3 0 2011  4  27119 11.4  3 1
390 0 2011  8 658931  8.7 37 1
 27 0 2011  4  66661  9.5 64 1
  3 0 2011  3  22760 15.3 50 1
  5 0 2011  5  25695 15.2 13 1
 12 0 2011  1  50044  8.3 23 1
 56 0 2011  0 102369  8.1 35 1
  4 0 2011  0  13862 11.8 14 1
end
label values fips fips
label def fips 1 "1 1", modify
label def fips 2 "1 3", modify
label def fips 3 "1 5", modify
label def fips 4 "1 7", modify
label def fips 5 "1 9", modify
label def fips 6 "1 11", modify
label def fips 7 "1 13", modify
label def fips 8 "1 15", modify
label def fips 9 "1 17", modify
label def fips 10 "1 19", modify
label def fips 11 "1 21", modify
label def fips 12 "1 23", modify
label def fips 13 "1 25", modify
label def fips 14 "1 27", modify
label def fips 15 "1 29", modify
label def fips 16 "1 31", modify
label def fips 17 "1 33", modify
label def fips 18 "1 35", modify
label def fips 19 "1 37", modify
label def fips 20 "1 39", modify
label def fips 21 "1 41", modify
label def fips 22 "1 43", modify
label def fips 23 "1 45", modify
label def fips 24 "1 47", modify
label def fips 25 "1 49", modify
label def fips 26 "1 51", modify
label def fips 27 "1 53", modify
label def fips 28 "1 55", modify
label def fips 29 "1 57", modify
label def fips 30 "1 59", modify
label def fips 31 "1 61", modify
label def fips 32 "1 63", modify
label def fips 33 "1 65", modify
label def fips 34 "1 67", modify
label def fips 35 "1 69", modify
label def fips 36 "1 71", modify
label def fips 37 "1 73", modify
label def fips 38 "1 75", modify
label def fips 39 "1 77", modify
label def fips 40 "1 79", modify
label def fips 41 "1 81", modify
label def fips 42 "1 83", modify
label def fips 43 "1 85", modify
label def fips 44 "1 87", modify
label def fips 45 "1 89", modify
label def fips 46 "1 91", modify
label def fips 47 "1 93", modify
label def fips 48 "1 95", modify
label def fips 49 "1 97", modify
label def fips 50 "1 99", modify
label def fips 51 "1 101", modify
label def fips 52 "1 103", modify
label def fips 53 "1 105", modify
label def fips 54 "1 107", modify
label def fips 55 "1 109", modify
label def fips 56 "1 111", modify
label def fips 57 "1 113", modify
label def fips 58 "1 115", modify
label def fips 59 "1 117", modify
label def fips 60 "1 119", modify
label def fips 61 "1 121", modify
label def fips 62 "1 123", modify
label def fips 63 "1 125", modify
label def fips 64 "1 127", modify
label def fips 65 "1 129", modify
label def fips 66 "1 131", modify
label def fips 67 "1 133", modify

[Clyde Schechter]

What you describe looks basically sound. In the example data you show, time0_rest_red = 0 for all observations, and year is restricted to 2010 and 2011. So it is not possible from your example to verify that time0_rest_red shows the kind of variation to be expected (stays constant at 0 in some jurisdictions, and changes from 0 before to 1 after some year in others). But if in your full data set it behaves that way, then your setup looks basically sound.

I do wonder about the inclusion of totpop as a covariate in your model. My instinct is to say that this would be better modeled as an -exposure()- variable. But that's a substance question--you might want to discuss that with your dissertation advisor.

Another substance issue to discuss with your advisor: the jurisdictions that had unrestricted practice all along does not strike me as an appropriate control group for changing from restricted to reduced. I would imagine that there is something very different in the environment of always-full-scope jurisdictions that makes the dynamics of the np workforce different. Nevertheless, if you conclude that it is appropriate to include this second control group, doing so is not difficult. Just include them in the estimation sample and make sure that time0_rest_red = 0 for these jurisdictions at all times. No additional variables are required here (because the variable that would distinguish the two control groups is time-invariant and, consequently, would be omitted in the fixed-effects model anyway).

Another substance issue to discuss with your advisor: it would surprise me if the change in scope of practice laws resulted in an instantaneous change in the number of np's in the jurisdiction. But that is what your model implies. I would consider having the effect delayed at least by one year.

[Oscar Ozfidan]

Clyde Schechter has touched on some of the things I have been thinking too. In addition to everything he said, I would familiarize myself with "Granger Causality test". Also, the state health boards usually set up a delayed effective date for significant changes like the ones you are considering. I would reach out to state board of nursing or of NPs to see if there was any such changes occurred. Investigating the lag between the policy effective date and actual results would be a very good idea as Clyde mentioned already. One of the the reason is that such policy changes would not immediately effect NPs near graduation but may those who are making decisions about becoming an NP. Also, you may add other control group and further distinguish it as a third category then use i.time0_rest_red in your model to investigate if that makes any difference. In any event I would use factor notation (i.time0_rest_red) even with only two categories. I also tend to think that you may want to try to control the number of establishments that hire NPs in each jurisdiction. Number of establishments by fips codes are relatively easy to access from your state or federal department of labor. If a new hospital pooped up in a given year, you dont want that skew your results. I also have second thoughts about using the unemployment rate as a control variable and tend to think that the number of unemployed in a country may be better suited. The reason is there is a natural unemployment rate that is the theoretical minimum (part of unemployment you have because of job changes etc that you will never be able to get rid of even in a perfect economy) . As you approach that natural minimum one percent reduction may not be the same as one percent reduction at a higher level. (e.g. 1% change from 5 to 4 % vs 1% change from 8 to 7%). The state board may also readily have data on license by examination vs license by endorsement. That information may help you enhance your study in terms of separating those who are coming to the stata from other states which I think may be more responsive to the policy changes of the kind you are interested in.

[Tammie Jones]

Good afternoon, sir. the time0_rest_red should = 0 for all county/states where the law does not change. For the other county/states it changes to = 1 in the year the policy was effective. I also have 2 lag variables for time, in which, time+1yr or +2yr, to study the effect 1 and 2 years after the law is effective. I have had the time0_rest_red variable dropped because it is constant within group and fips omitted due to collinearity. I'm also running this model with the option cluster(state_fips_code), since the policy change is by state, to compare the standard errors between the models. This is still running so no results yet.

I'm not familiar with an -exposure()- variable. The idea on inclusion of totpop is that as the greater the population, the greater demand for services will have an effect on the supply of providers (in this case nurse practitioners). Also not included in the model yet, is the number of nursing schools producing nurse practitioners.

Thanks for your thoughts about the additional potential control variable. I currently have a separate model for county/states that transition from reduced to full practice group, but have not included the always full practice (at least during the study period) in either model.

[Tammie Jones]

@Oscar Ozfidan I tried to simplify my post here so I didn't include all covariates from my study. I have also included the count of physicians, physician assistants, and the number of clinics. Clinics is very important, because people need a place to practice and the trend is not going the way of private practice. The idea on including unemployment is about providers interest in going to practice in areas of the country where the economies aren't good. Unemployment is one factor that represents this concept.

Thanks for your other suggestions about the policy timing. I have an annual legislative update from the American Association of Nurse Practitioners that addresses most of the policy changes and complexities by state. Also another researcher just published a paper summarizing the legislative changes by nuance of the state practice acts. Had this not been available I would still be collecting data = )

[Oscar Ozfidan]

Basically you do not need a separate exposure variable. You would be treating population as an exposure variable, an option available in Poisson.

[Oscar Ozfidan]

I am personally interested in what you eventually find out from your study in this particular area. So, when you are done, I would be very interested in reading your research if you get a chance to send a copy to me at this forum.