Cramer (1987) test for comparing adjusted R-squared of models

[Der Basil]

Hello Respected Members,

I humbly request for your assistance. Can somebody guide me as to how to apply crammer (1987) test for comparing adjusted R-squared of independent samples or any other test meant for the same purpose?

I would be most grateful.

[Maria Boutchkova]

Hi,

I am not able to find quickly the exact form of Cramer's test, but the internet tells me that there are improvements on it in Lien and Vuong (1987) and Vuong (1989).

Suppose you have the exact formula for the test. You could compute it yourself using stored results after the reg command.

For example:

sysuse auto, clear

qui reg price mpg if foreign == 0
local r_sq_dom = `e(r2_a)'
display "Adj Rsq for dometic cars: " %9.4f `r_sq_dom'

qui reg price mpg if foreign == 1
local r_sq_for = `e(r2_a)'
display "Adj Rsq for foreign cars: " %9.4f `r_sq_for'

Once you have the test computed and stored in the local macro `t_test' and the respective degrees of freedom in the macro `df', you ask Stata to produce the p-value like this:

display "The right-tail (upper-tail) p-value of a t-statistic " ///
"that has `df' degrees of freedom" ///
", and calculated sample value `t_test' is: " ///
%9.4f ttail(`df', `t_test')

Maria

[Der Basil]

Hello Maria,
I seem to be lost by your submission. the reason being that i am new to STATA. I copied them and pressed the send button and i got the message "unrecognized command". Can you help?

[Steve Samuels]

Maria, please give full references, as the FAQ request.

[Maria Boutchkova]

Der, I cannot be sure what went wrong for you. The code works fine for me. Could you specify your version of Stata.


Steve, here are my references:

Lien, D. and Vuong, Q. H. (1987) Selecting the Best Linear Regression Model: A Classical Approach, Journal of Econometrics, 35, pp. 3-23.
Vuong, Q. H. (1989) Likelihood Ratio Test for Model Selection and Non-Nested Hypotheses, Econometrica, 57, pp. 307-333.

[Der Basil]

hello Maria,

i am using Stata 11

[Der Basil]

Hello Maria,

attached is my data in excel format. i am interested in comparing the adjusted r-squared of the following models from the two independent samples using the Vuong (1989) test

regress d.mps d.bvp d.eps
regress d.mps d.bvp d.dps d.dre
regress d.mps d.bvp d.dps

can you help? or can i have a skype lesson with you for some few minutes. You could help me figure it out. i am beginner.

thanks

[Abida Zainab]

Hi basil

Have u upgraded stata 11 after installation?

[Der Basil]

Hello Abida Zainab,

I install programs using the ssc command which are not found in stata 11. Is that the upgrading?

Thank you

[Rich Goldstein]

no, see "help update"

please read the FAQ to see why posting excel files is a bad idea and to see why full citations are MUCH better than just author and year (which is what you did)

[Der Basil]

hello Rich Goldstein,
i am reffering to this test "Vuong, Q. H. (1989) Likelihood Ratio Test for Model Selection and Non-Nested Hypotheses, Econometrica, 57, pp. 307-333".

regress dmps dbvp deps
regress dmps dbvp ddps d.dre
regress dmps dbvp ddps


below are the two independent datasets

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte number int year float(dmps dbvp deps ddps dre)
1 2006 . . . . .
1 2007 0 .0009999983 .002 0 .001
1 2008 .009999998 .0030000024 .003 0 .003
1 2009 -.019999996 -.012000002 -.013 0 -.004
1 2010 -.06 -.022 -.014 0 0
2 2005 . . . . .
2 2006 -.02 -.029 -.005999999 0 0
2 2007 .01 .0029999986 .031999998 0 .003
2 2008 .07 .084 -.024 0 -.003
2 2009 -.009999998 -.013999999 .006999999 0 0
2 2010 -.010000005 -.009000003 .005000001 0 0
2 2011 0 -.008000001 -.008000001 0 0
3 2010 . . . . .
3 2011 0 .005999999 .005 0 .005
3 2012 .010000002 .001000002 -.003 0 -.003
3 2013 -.010000002 .006999999 .003 .01 -.004
3 2014 -.02 .0009999983 -.006 0 0
4 2006 . . . . .
4 2007 -.03000003 -.086 -.117 0 -.032
4 2008 -.13 .304 .028 0 0
4 2009 -.09 -.123 -.107 0 0
4 2010 -.16 -.05399999 .08 0 0
4 2011 -.12 -.032000005 .044 0 0
4 2013 . . . . .
5 2007 . . . . .
5 2008 .03 .009 .0010000002 0 .0010000002
5 2009 -.03 .011999998 .005 0 .005
5 2010 .04000001 .013999999 -.0020000003 0 -.0020000003
5 2012 . . . . .
5 2013 -.010000005 .001000002 -.009 0 -.009
6 2005 . . . . .
6 2006 0 -.002000004 .015 0 .015
6 2007 .01999998 .161 .006 0 .006
6 2008 .31 .09400001 .121 0 .121
6 2009 -.38 .009000003 -.095 0 -.095
6 2010 .31 .04399997 .03 .04 -.009999998
6 2011 .8800001 .222 .198 0 .198
6 2012 1.57 .317 .111 .010000002 .101
6 2013 -.1600001 .08899999 -.219 .02 -.239
6 2014 1.46 .319 .186 .009999998 .176
7 2005 . . . . .
7 2006 0 .0020000003 .003 0 .003
7 2007 0 -.009000003 .007 0 .007
7 2008 -.010000005 .014000002 .0029999996 0 .0029999996
7 2009 0 -.0020000003 -.007999999 0 -.007999999
7 2010 -.05 .016999997 .013 0 .013
7 2011 .009999998 .025000006 .007000001 0 .007000001
7 2012 .04 .137 .018000001 0 .018000001
7 2013 0 -.06200001 -.034 0 -.034
7 2014 -.04 .006999999 .0020000003 .01 -.007999999
8 2010 . . . . .
8 2011 -.03 0 -.001 0 -.001
8 2012 0 -.01 -.011 0 -.001
8 2013 0 -.003 -.005 0 0
8 2014 -.01 -.023 -.015 0 0
9 2005 . . . . .
9 2006 .06 .020000003 -.0019999985 0 -.0019999985
9 2007 .16 .038 .009 0 .009
9 2008 .19000003 .04900001 .022 0 .022
9 2009 .48 .116 .069000006 0 .069000006
9 2010 1.3 .143 .034999996 0 .034999996
9 2011 -.22000003 .098 -.001000002 0 -.001000002
10 2008 . . . . .
10 2009 -.1 .013000004 .003999999 0 .003999999
10 2010 .1 .018 .0040000007 .01 -.006
10 2011 .03999999 .025000006 .007000001 0 .007
10 2012 .67 .027999997 .004999999 0 .005000001
10 2013 -.22000003 .049 .018 0 .018000001
10 2014 .26 .07000001 .025 0 .025
11 2006 . . . . .
11 2007 0 -.0039999997 -.004 0 0
11 2009 . . . . .
11 2010 0 -.007999999 -.0020000003 0 0
11 2011 -.010000002 -.018000001 -.0009999992 0 0
11 2012 0 -.006000001 .0029999996 0 0
11 2013 0 -.005999997 0 0 0
12 2005 . . . . .
12 2006 .13999993 .049 .025000006 0 .025000006
12 2007 1.46 .03100002 -.006000005 0 -.006000005
12 2008 -1.23 .03 -.001000002 0 -.001000002
12 2009 .26 .00999999 -.013999999 0 -.013999999
12 2010 -.00999999 -.11199999 -.097 .01 -.069
12 2011 .95 .0039999783 .031 -.01 .003
12 2013 . . . . .
12 2014 -1.0899999 -.05799997 -.127 0 -.086
13 2005 . . . . .
13 2006 -.04000001 .011000007 -.0010000011 0 -.0010000011
13 2007 0 .08999999 .008 0 .008
13 2008 0 .015000015 .006999999 0 .006999999
13 2009 -.029999986 .04699999 -.01 0 -.01
13 2010 -.07000001 -.032999992 .017 0 .017
13 2011 0 .06999999 .031000003 0 .031000003
13 2012 .12 .448 .05599999 .01 .046
13 2013 .12 -.036000013 -.144 0 -.114
13 2014 -.15 -.07999998 -.05 0 0
14 2005 . . . . .
14 2006 -.02000001 -.0040000007 .006 0 0
14 2007 -.04000001 .0010000002 .003 0 .001
14 2008 -.11 .005 .003 0 .003
14 2011 . . . . .
14 2012 -.06999999 .0029999986 -.037 0 -.037
14 2013 -.010000005 -.019 -.039 .02 -.021
14 2014 -.05 -.054 -.035 -.01 0
15 2005 . . . . .
15 2006 0 .012999997 .007 0 .007
15 2007 .009999998 .019 .006999999 0 .006999999
15 2008 .06000001 .019 .0010000002 0 .0010000002
15 2009 0 .003000006 -.015999999 0 -.015999999
15 2011 . . . . .
15 2012 -.04000001 0 -.031999998 0 -.031999998
15 2013 .4 .04699999 .041 0 .041
15 2014 -.18999997 -.013999999 -.057 0 -.047
16 2006 . . . . .
16 2007 .01999998 .011000007 .039 0 .039
16 2008 .25 -.007999986 -.016999997 0 -.016999997
16 2009 -.06 .04099998 .06 0 .06
16 2010 1.09 .05400002 .072 .06 .01200001
16 2011 1.33 .05699998 .014000013 -.03 .044
16 2012 .26 .12 .070999995 .11 -.039
17 2009 . . . . .
17 2010 -.020000003 -.022 -.0020000003 0 0
17 2011 -.02 -.014 .009 0 0
17 2012 -.010000002 -.010000002 .0030000005 0 0
17 2013 0 -.009 -.0010000002 0 0
17 2014 -.01 .02 .015999999 0 .005
18 2005 . . . . .
18 2006 -.03999996 .02700001 -.004999995 0 -.004999995
18 2007 .73 .349 .01699999 0 .01699999
18 2008 1.27 .097 .147 0 .147
18 2009 -.1099999 -.148 -.28599998 0 -.28599998
end

second dataset

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte number int year float(dmps dbvp deps ddps dre)
1 2006 . . . . .
1 2007 0 .0009999983 .002 0 .001
1 2008 .009999998 .0030000024 .003 0 .003
1 2009 -.019999996 -.012000002 -.013 0 -.004
1 2010 -.06 -.022 -.014 0 0
2 2005 . . . . .
2 2006 -.02 -.029 -.005999999 0 0
2 2007 .01 .0029999986 .031999998 0 .003
2 2008 .07 .084 -.024 0 -.003
2 2009 -.009999998 -.013999999 .006999999 0 0
2 2010 -.010000005 -.009000003 .005000001 0 0
2 2011 0 -.008000001 -.008000001 0 0
3 2010 . . . . .
3 2011 0 .005999999 .005 0 .005
3 2012 .010000002 .001000002 -.003 0 -.003
3 2013 -.010000002 .006999999 .003 .01 -.004
3 2014 -.02 .0009999983 -.006 0 0
4 2006 . . . . .
4 2007 -.03000003 -.086 -.117 0 -.032
4 2008 -.13 .304 .028 0 0
4 2009 -.09 -.123 -.107 0 0
4 2010 -.16 -.05399999 .08 0 0
4 2011 -.12 -.032000005 .044 0 0
4 2013 . . . . .
5 2007 . . . . .
5 2008 .03 .009 .0010000002 0 .0010000002
5 2009 -.03 .011999998 .005 0 .005
5 2010 .04000001 .013999999 -.0020000003 0 -.0020000003
5 2012 . . . . .
5 2013 -.010000005 .001000002 -.009 0 -.009
6 2005 . . . . .
6 2006 0 -.002000004 .015 0 .015
6 2007 .01999998 .161 .006 0 .006
6 2008 .31 .09400001 .121 0 .121
6 2009 -.38 .009000003 -.095 0 -.095
6 2010 .31 .04399997 .03 .04 -.009999998
6 2011 .8800001 .222 .198 0 .198
6 2012 1.57 .317 .111 .010000002 .101
6 2013 -.1600001 .08899999 -.219 .02 -.239
6 2014 1.46 .319 .186 .009999998 .176
7 2005 . . . . .
7 2006 0 .0020000003 .003 0 .003
7 2007 0 -.009000003 .007 0 .007
7 2008 -.010000005 .014000002 .0029999996 0 .0029999996
7 2009 0 -.0020000003 -.007999999 0 -.007999999
7 2010 -.05 .016999997 .013 0 .013
7 2011 .009999998 .025000006 .007000001 0 .007000001
7 2012 .04 .137 .018000001 0 .018000001
7 2013 0 -.06200001 -.034 0 -.034
7 2014 -.04 .006999999 .0020000003 .01 -.007999999
8 2010 . . . . .
8 2011 -.03 0 -.001 0 -.001
8 2012 0 -.01 -.011 0 -.001
8 2013 0 -.003 -.005 0 0
8 2014 -.01 -.023 -.015 0 0
9 2005 . . . . .
9 2006 .06 .020000003 -.0019999985 0 -.0019999985
9 2007 .16 .038 .009 0 .009
9 2008 .19000003 .04900001 .022 0 .022
9 2009 .48 .116 .069000006 0 .069000006
9 2010 1.3 .143 .034999996 0 .034999996
9 2011 -.22000003 .098 -.001000002 0 -.001000002
10 2008 . . . . .
10 2009 -.1 .013000004 .003999999 0 .003999999
10 2010 .1 .018 .0040000007 .01 -.006
10 2011 .03999999 .025000006 .007000001 0 .007
10 2012 .67 .027999997 .004999999 0 .005000001
10 2013 -.22000003 .049 .018 0 .018000001
10 2014 .26 .07000001 .025 0 .025
11 2006 . . . . .
11 2007 0 -.0039999997 -.004 0 0
11 2009 . . . . .
11 2010 0 -.007999999 -.0020000003 0 0
11 2011 -.010000002 -.018000001 -.0009999992 0 0
11 2012 0 -.006000001 .0029999996 0 0
11 2013 0 -.005999997 0 0 0
12 2005 . . . . .
12 2006 .13999993 .049 .025000006 0 .025000006
12 2007 1.46 .03100002 -.006000005 0 -.006000005
12 2008 -1.23 .03 -.001000002 0 -.001000002
12 2009 .26 .00999999 -.013999999 0 -.013999999
12 2010 -.00999999 -.11199999 -.097 .01 -.069
12 2011 .95 .0039999783 .031 -.01 .003
12 2013 . . . . .
12 2014 -1.0899999 -.05799997 -.127 0 -.086
13 2005 . . . . .
13 2006 -.04000001 .011000007 -.0010000011 0 -.0010000011
13 2007 0 .08999999 .008 0 .008
13 2008 0 .015000015 .006999999 0 .006999999
13 2009 -.029999986 .04699999 -.01 0 -.01
13 2010 -.07000001 -.032999992 .017 0 .017
13 2011 0 .06999999 .031000003 0 .031000003
13 2012 .12 .448 .05599999 .01 .046
13 2013 .12 -.036000013 -.144 0 -.114
13 2014 -.15 -.07999998 -.05 0 0
14 2005 . . . . .
14 2006 -.02000001 -.0040000007 .006 0 0
14 2007 -.04000001 .0010000002 .003 0 .001
14 2008 -.11 .005 .003 0 .003
14 2011 . . . . .
14 2012 -.06999999 .0029999986 -.037 0 -.037
14 2013 -.010000005 -.019 -.039 .02 -.021
14 2014 -.05 -.054 -.035 -.01 0
15 2005 . . . . .
15 2006 0 .012999997 .007 0 .007
15 2007 .009999998 .019 .006999999 0 .006999999
15 2008 .06000001 .019 .0010000002 0 .0010000002
15 2009 0 .003000006 -.015999999 0 -.015999999
15 2011 . . . . .
15 2012 -.04000001 0 -.031999998 0 -.031999998
15 2013 .4 .04699999 .041 0 .041
15 2014 -.18999997 -.013999999 -.057 0 -.047
16 2006 . . . . .
16 2007 .01999998 .011000007 .039 0 .039
16 2008 .25 -.007999986 -.016999997 0 -.016999997
16 2009 -.06 .04099998 .06 0 .06
16 2010 1.09 .05400002 .072 .06 .01200001
16 2011 1.33 .05699998 .014000013 -.03 .044
16 2012 .26 .12 .070999995 .11 -.039
17 2009 . . . . .
17 2010 -.020000003 -.022 -.0020000003 0 0
17 2011 -.02 -.014 .009 0 0
17 2012 -.010000002 -.010000002 .0030000005 0 0
17 2013 0 -.009 -.0010000002 0 0
17 2014 -.01 .02 .015999999 0 .005
18 2005 . . . . .
18 2006 -.03999996 .02700001 -.004999995 0 -.004999995
18 2007 .73 .349 .01699999 0 .01699999
18 2008 1.27 .097 .147 0 .147
18 2009 -.1099999 -.148 -.28599998 0 -.28599998
end

[Rich Goldstein]

as the FAQ requests, please use dataex (search dataex to find and download) for showing data sets; use "hsearch vuong" to find a number of places where vuong tests are implemented in Stata - I don't know whether any of these are what you are referring to as Cramer's (or Crammer's) test

[Maria Boutchkova]

Dar, I looked at Vuong (1989) (full reference above) and he derives a Likelihood ratio test for comparing two models. However, because you use OLS, see Maarten Buis in this link explains that in Stata you should use -test in this case. http://stats.stackexchange.com/quest...ression-models

[Der Basil]

hello Maria,

thank you very much. the Vuong, (1989) worked perfectly. i didn't just know how to apply it. i will like to clarify some issues about it.

1. will the degrees of freedom (df) be based on the observations when "foreign==1" , or the total number of observations (foreign==1 + foreign==0)?

2. i choose 2.0 for the t_test. is it reasonable?

thank you very much

[Der Basil]

hello Maria,

I have sorted it out using the Vuong (1989) command you provided. With 200 df, the corresponding t_stats is 1.65.

Thank you very much. You have been helpful. God bless you a million fold

Basil