regress help requested

[Sunita Rao]

I have a dataset in which all the dependent variables are for the years 2020 & 2021. I have 4 dependent variables, which are used in 4 different regressions. The independent variables are for the years 2018, 2019, 2020 & 2021.

Here is a sample of the dependent variables:

* Example generated by -dataex-. For more info, type help dataex
clear
input int year float(raw_return_ AbRet) double Vol1Yr_ float idio_volatility
2018         .         .       . .023706943
2019         .         .       . .023706943
2020         .         . 36.2264 .023706943
2021 -.3217149 -9.685585 28.7132 .023706943
2018         .         .       . .023706943
2019         .         .       . .023706943
2020         .         . 36.2264 .023706943
2021 -.3217149 -9.685585 28.7132 .023706943
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
2018         .         .       . .017640421
2019         .         .       . .017640421
2020  .0612004 -20.10798 28.3259 .017640421
2021         .         . 11.9041 .017640421
end

I have a plenty of independent variables. Here is a sample:

input int year byte(ESGScrFY ENVScrFY SocialScrFY GOVScrFY) float(size ROE profitability cash_over_assets stdebt_over_assets mtb) byte neg_mtb float HistVolatility
2018 13 0 4 26 6.073982 .021759655 .3056392 .3541455 .12199316 13.067905 0 1040.8689
2019 16 0 2 28 6.235095 .02212184 .2627229 .2828731 .09094698 12.02771 0 1040.8689
2020 20 0 8 32 6.534146 .023738274 .28146 .3461379 .1122328 13.204992 0 1040.8689
2021 24 16 17 31 6.548369 .01363123 .22036627 .3311015 .10458087 18.168388 0 1040.8689
2018 13 0 4 26 6.073982 .021759655 .3056392 .3541455 .12199316 13.067905 0 1040.8689
2019 16 0 2 28 6.235095 .02212184 .2627229 .2828731 .09094698 12.02771 0 1040.8689
2020 20 0 8 32 6.534146 .023738274 .28146 .3461379 .1122328 13.204992 0 1040.8689
2021 24 16 17 31 6.548369 .01363123 .22036627 .3311015 .10458087 18.168388 0 1040.8689
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 .
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 .
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 .
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 .
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 .
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 .
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 .
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 .
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 .
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 .
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 .
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 .
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 .
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 .
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 .
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 .
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 .
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 .
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 .
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 .
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 199.5806
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 199.5806
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 199.5806
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 199.5806
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 199.5806
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 199.5806
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 199.5806
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 199.5806
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 199.5806
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 199.5806
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 199.5806
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 199.5806
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 198.37877
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 198.37877
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 198.37877
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 198.37877
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 198.37877
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 198.37877
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 198.37877
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 198.37877
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 198.37877
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 198.37877
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 198.37877
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 198.37877
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 198.37877
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 198.37877
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 198.37877
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 198.37877
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 198.37877
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 198.37877
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 198.37877
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 198.37877
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 180.287
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 180.287
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 180.287
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 180.287
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 180.287
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 180.287
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 180.287
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 180.287
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 180.287
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 180.287
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 180.287
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 180.287
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 180.287
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 180.287
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 180.287
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 180.287
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 180.287
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 180.287
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 180.287
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 180.287
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 170.62106
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 170.62106
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 170.62106
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 170.62106
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 170.62106
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 170.62106
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 170.62106
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 170.62106
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 170.62106
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 170.62106
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 170.62106
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 170.62106
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 170.62106
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 170.62106
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 170.62106
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 170.62106
2018 76 83 73 75 9.837869 .08979048 .0126145 .020834863 .8490396 1.8775734 0 170.62106
2019 75 86 78 71 9.90103 .09026607 .011974652 .016592776 .8493556 1.830828 0 170.62106
2020 78 91 75 77 9.893396 .09819403 .009640835 .07133553 .8490928 1.442319 0 170.62106
2021 77 89 74 76 10.083617 .09861282 .012845165 .065547965 .8628278 1.7681098 0 170.62106
end
[/CODE]

I want to perform a regress where the dependent variable is from 2020 & 2021, but the independent variables are from 2018 & 2019. The following command gives me an error, since the dependent variables do not exist for those years.

regress raw_return_ ESGScrFY ENVScrFY SocialScrFY GOVScrFY size ROE profitability cash_over_assets stdebt_over_assets mtb neg_mtb HistVolatility if year==2018|year==2019, vce(cluster num_CIQ_ID)
no observations
r(2000);

How can I perform this regression where my dependent variable is from 2020 & 2021 and my independent variables are from 2018 & 2019. I would be very grateful for some advice.

[Jared Greathouse]

Impossible.

It would be a little like saying set A= 1 2 3 and set B = 4 5 6

And you wanna take the average of all numbers greater than 1000 in these sets. There are no observations like that, so this can't work, and you wouldn't want it to

[Sunita Rao]

Dear Dr. Greathouse,

Thank you so much for your input. I greatly appreciate your quick response.

[Clyde Schechter]

Well, I'm not sure exactly what is meant by "How can I perform this regression where my dependent variable is from 2020 & 2021 and my independent variables are from 2018 & 2019." If you mean that you would like to do a regression where the dependent variables from 2020 and 2021 are predicted by the independent variables in 2018 and 2019, respectively, then this is just a matter of using the second lag of the independent variables in the regression. However, to even make sense of the concept of a lagged variable, there must be some panel identifier variable, and none is found in the example data. The data does, however, look like it comes from a panel data set with the panel identifier, for some reason, deleted/omitted.

It also appears, however, that for most of your dependent variables, they are available only for 2020 or 2021 and not both. For those, it is even less clear what you might want to do here and the various things I can think of get pretty complicated pretty fast.

In short, I think you need to spell out more carefully what you are trying to do here. It may or may not be possible.

[Sunita Rao]

Dear Prof. Schechter,

Yes, that is exactly correct. I would like to do a regression where the dependent variables from 2020 and 2021 are predicted by the independent variables in 2018 and 2019.

I will declare my dataset to be a panel dataset with the xtset command asap. Thank you very much for this suggestion.

[Sunita Rao]

The xtset command works when I provide it with the identifier variable i.e. xtset num_CIQ_ID.

But, it does not work when I give the following command:

xtset num_CIQ_ID year
repeated time values within panel
r(451);

The variable year is as mentioned above.

I would be very grateful for some advice on how to proceed. What am I doing wrong with the time variable?

[Clyde Schechter]

You have to identify the surplus observations.

duplicates tag num_CIQ_ID year, gen(flag)
browse if flag

That will show you all the observations where a num_CIQ_ID has more than one observation for the same year. Then you have to figure out why they are there: they shouldn't be, so something went wrong in creating this data set. And you have to then figure out how to fix whatever problem led to it. There is no point proceeding with anything until you fix this corrupted data set.

[Sunita Rao]

Thank you so much, Prof. Schechter. I will get started with fixing my dataset.

[Jared Greathouse]

I don't quite have my Ph.D yet! But I will, in 3 years. But yes, the issue is fixing the duplicates as Clyde suggests.

[Sunita Rao]

Jared,

Thank you!

Good luck with your Ph.D.!

[Sunita Rao]

Dear Prof. Schechter,

Thank you again for your suggestion.

I dropped the surplus copies and was able to do the xtset regressions.