[Clyde Schechter]

OK. You can -collapse- your data set to get the variance for each indicator in each clinic in each time period:

collapse (sd) Achievement, by(Indicator Treated Time)
gen var_Achievement = Achievement*Achievement
regress var_Achievement i.Treated##i.Time

[Stephen Butler]

Great thank you Clyde.

[Stephen Butler]

Hello again -
Regarding the same analysis as above (a DiD on financial incentive withdrawal); I'd also like to test the relationship between the relative drop in quality indicator performance, and its financial value to the GP practice.

So if this is my data:

Practice Indicator Achievement Treated Time Points
W95009 AST004 85.714286 1 0 5
W95009 COPD002 95.876289 1 0 6
W95009 COPD003 95.918367 1 0 5
W95009 COPD003 100 1 0 5
W95009 DM002 87.665198 1 0 10
W95009 DM002 73.827534 1 0 10

The 'point' value of an individual quality indicator (eg AST004, COPD003, DM002) is the same every year, and for every GP practice. The greater the number of points, the more that indicator is worth financially to a GP practice. So how would I establish, in Stata, whether there's a relationship between the relative change in quality performance for an indicator (relative to the controls) and the points value of that indicator? Many thanks.

[Clyde Schechter]

I don't understand your question. You say that the number of points for an indicator is the same every year, so it is not going to change regardless of what happens to quality performance. What am I missing here?

[Stephen Butler]

Sorry I explained badly. Please forget I said the points stay the same every year. Say you have lots of indicators which are financially incentivised (and the ‘points’ variable shows what they’re worth, financially) - how do you calculate the relationship, if any, between the extent in the performance drop after removing the financial incentive and the number of ‘points’.

for example, when you remove financial incentives for an indicator worth a lot of points, do you see a bigger drop than for an indicator that isn’t worth much?

using DiD, I can work out the performance change after incentive removal for each indicator (relative to the control) - but I’d now like to see if the extent of that performance change is related to the value of the original incentive. Thanks for your help.

[Clyde Schechter]

So create a data set containing your estimates of the performance change for each indicator, one observation per indicator, and have as a second variable the value of the original incentive. Then calculate the correlation between them. (or, if you like, regress the performance change on the value of the original incentive.

[Stephen Butler]

That makes sense, thanks for your advice Clyde.

[Stephen Butler]

Hello again
Regarding your helpful advice above (about calculating the DiD of the variance), I did the following:

collapse (sd) Achievement, by (Treat_ind Time) -- calculates SD for each indicator, for each year (where the control indicators are coded '0' and the rest are coded '1-26')

To get the relative difference in SD, between each of the 'treated' indicators, and the controls, after removing the incentive, I did:

. reg sd_achievement Time##Treat_ind


The results I get look correct (the SD increases for nearly all the indicators for which financial incentives were removed); however, STATA does not give confidence intervals:

I get:

sd_achievement | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------------+----------------------------------------------------------------
Time |
1 | -.9744356 . . . . .

|
Time#Treat_ind |
1 1 | -2.944269 . . . . . [for indicator 1, compared to control]
1 2 | -1.238244 . . . . . [for indicator 2, compared to control]
1 3 | 2.289503 . . . . . [for indicator 3, compared to control]
1 4 | 4.75876 . . . . .
1 5 | 8.141209 . . . . .
1 6 | 3.351372 . . . . .
1 7 | 5.592581 . . . . .
1 8 | 5.040912 . . . . .
1 9 | 1.166605 . . . . .
1 10 | 10.12798 . . . . .
1 11 | 1.767805 . . . . .
1 12 | 2.099067 . . . . .
1 13 | .8830405 . . . . .
1 14 | 3.744449 . . . . .
1 15 | 15.88724 . . . . .

etc.

Is there a reason why it wouldn't calculate CIs?
Many thanks.

[Farza Chow]

Hi;
This is what my data looks like:
Selection showup finished firmage employee
1 1 0 5 10000
1 0 0 10 50000
1 1 1 8 3000


I am trying to see if the treatment effect of the program varies by firm
age and employee numbers.
To do this, would I do three-way interactions?

showup*finished*firmage
showup*finished*employee

Please let me know if you have any clarification questions.

[Farza Chow]

Hi;
I am running DID model. This is what my data looks like:
Selection showup finished firm_age employee
1 1 0 5 10000
1 0 0 10 50000
1 1 1 8 3000


I am trying to see if the treatment effect of the program varies by firm age and employee numbers.

To do this, would I do three-way interactions?

showup##finished##firmage
showup##finished##employee

Please Help!

[Zihao Chen]

Hi all,

If anyone still there. May I ask a question about the interpreting Stata's DiD output.

where ESGlow is the ESG score below the average, and the post is a time
And what I need to do for the diff in diff regression?

[Joro Kolev]

You do not say how you generated your variables, but from the names I guess that your DiD estimator is postesglow, that is -13028.

[marco derrico]

Dear Clyde, thank you for the very valuable inputs you provide. Here my question to you.

I have a multivalued treatment. I have three districts, Torit receiving Leg1 and Leg3 of the same project; Lopa and Ikotos receiving Leg2. Then I have Control group, not receiving intervention at all.
I run reg analysis on the different legs using normal DD (T/C) specification with the three legs. Now, as robustness check, I would like to do a DDD using Torit vs Lopa and Ikotos. Therefore, mimicking what you wrote above, I created:
gen byte group = 2 if county == 7
replace group = 1 if county == 6 | county == 2
replace group = 0 if group == .

and then run reg like this:
reg outcome i.group##i.time sample_type time hhsize i.livelihood i.respondent_sex

note: time omitted because of collinearity

Source | SS df MS Number of obs = 1,893
-------------+---------------------------------- F(16, 1876) = 22.87
Model | 5549.01804 16 346.813627 Prob > F = 0.0000
Residual | 28448.8607 1,876 15.1646379 R-squared = 0.1632
-------------+---------------------------------- Adj R-squared = 0.1561
Total | 33997.8788 1,892 17.9692805 Root MSE = 3.8942

---------------------------------------------------------------------------------------
outcome| Coef. Std. Err. t P>|t| [95% Conf. Interval]
----------------------+----------------------------------------------------------------
group |
1 | 1.818588 .2842156 6.40 0.000 1.261176 2.376
2 | -.2574121 .3543982 -0.73 0.468 -.9524682 .437644
|
1.time| -1.545975 .309598 -4.99 0.000 -2.153167 -.9387821
|
group#time|
1 1 | .5247082 .4441274 1.18 0.238 -.3463274 1.395744
2 1 | 3.210997 .4997635 6.43 0.000 2.230846 4.191148
|
sample_type | 2.096138 .2237756 9.37 0.000 1.657263 2.535013
time| 0 (omitted)
hhsize | .0704725 .0431657 1.63 0.103 -.0141853 .1551303
|
livelihood |
Agro-pastoralist | 1.991875 .6542994 3.04 0.002 .7086442 3.275106
Farmer | 1.300762 .6742905 1.93 0.054 -.0216768 2.6232
Fishing | 1.184525 1.731679 0.68 0.494 -2.211695 4.580744
Urban | 1.134312 .776895 1.46 0.144 -.3893572 2.657981
Entrepreneur | 4.567587 .8552253 5.34 0.000 2.890294 6.24488
Mixed | 2.015542 .8643045 2.33 0.020 .3204425 3.710641
Other | 1.362275 1.189725 1.15 0.252 -.9710492 3.6956
9 | .2822794 1.038636 0.27 0.786 -1.754723 2.319282
|
respondent_sex |
Female | 1.023254 .2267397 4.51 0.000 .5785654 1.467943
_cons | -3.669185 .6930335 -5.29 0.000 -5.028383 -2.309987
---------------------------------------------------------------------------------------

. margins i.group##i.time

Predictive margins Number of obs = 1,893
Model VCE : OLS

Expression : Linear prediction, predict()

---------------------------------------------------------------------------------------
| Delta-method
| Margin Std. Err. t P>|t| [95% Conf. Interval]
----------------------+----------------------------------------------------------------
group |
0 | -.8087333 .1593771 -5.07 0.000 -1.121308 -.4961583
1 | 1.208595 .1607436 7.52 0.000 .8933399 1.52385
2 | .1500642 .2231576 0.67 0.501 -.287599 .5877273
|
time|
0 | .35395 .1199491 2.95 0.003 .1187024 .5891976
1 | -.2376012 .1591548 -1.49 0.136 -.5497403 .0745378
|
group#time|
0 0 | -.223174 .1952356 -1.14 0.253 -.6060757 .1597277
0 1 | -1.769149 .2531619 -6.99 0.000 -2.265657 -1.27264
1 0 | 1.595414 .2047965 7.79 0.000 1.193761 1.997067
1 1 | .5741473 .2592398 2.21 0.027 .0657186 1.082576
2 0 | -.4805861 .2715274 -1.77 0.077 -1.013114 .0519414
2 1 | 1.184436 .343985 3.44 0.001 .5098027 1.85907
---------------------------------------------------------------------------------------

.
end of do-file

Am I doing right?
How shall I interpret the results?

thank you in advance and looking forward at reading your insights.

[Clyde Schechter]

Assuming that Torit county is coded 7, and Lopa and Ikotos are coded 2 and 6, the setup looks right.

That said, since you say your goal now is to contrast Lopa and Ikotos with Torit, it would be easier to interpret the outputs if you made group 1 or group 2 the base category instead of group 0. So I would re-run this as

reg outcome ib2.group##i.time sample_type hhsize i.livelihood i.respondent_sex

Then the coefficient for 1.group##1.time will be the difference in treatment effects you are now focusing on.

A couple of notes:

It seems that time is a 0/1 variable in your data. There is no reason to have continuous time included in your regression (between sample_type and hhsize). It gets omitted anyway, so no harm done. But it leads me to think you had something in mind when you put it there--whatever you had in mind, it isn't happening! (I've omtited it in the code above.)

If this is panel data (as opposed to serial cross sections), you should be using -xtreg-, not -regress- for this analysis.

In the future, when posting Stata output, please put it between code delimiters (just as I have here with the code I'm showing) so it will be well-aligned and easy to read. If you don't know how to use code delimiters, see Forum FAQ #12 for directions.

[marco derrico]

Dear Clyde
Thank you for the prompt response. Yes, you guessed right with the district-codes, and ok, I re-adjusted the routine as you indicated (incl xtreg).

As per the code delimiters, I apologize, didn't know. Actually, the codes in my post look almost illegible! Apologize!