K-fold cross validation code

[Kendal zhang]

local model "reghdfe ltrade lconflict_o lconflict_d lgdpcap_o lgdpcap_d lpop_o lpop_d lexchangerate_o lexchangerate_d d wto_o wto_d year_*, absorb(country_pair)"
local k `299'
crossfold `model', `k', r2


| Pseudo-R2
-------------+-----------
est1 | .1057278
est2 | .0775754
est3 | .1167998
est4 | .1132827
est5 | .0693004

The above is my code and outcome.my k is 299, but why do the result only show est1-5? and how can I evaluate the model through the result?

[Rich Goldstein]

your command "local k `299'" is not meaningful and does not do what you want and so the program steps back to its default of 5; why not just use 299 in your crossfold command if that is what you want? also, please tell people, as per the FAQ, where you got the -crossfold- command; see

h local

for how to define a local macro; note also that it is often worthwhile to check what is in your new macro by listing it; here you could have gone
mac li _k
to see what is in the macro

[Dimitriy V. Masterov]

Get rid of the quotes around 299. With quotes, you define k to contain the contents of local 299. This local does not exist, so the contents are empty, and k inherits that. When you evaluate k, the code default to 5 since there's nothing inside k.

[William Lisowski]

Looking at the output of help crossfold, it seems likely that using macros, as you have done, will in any event not result in the syntax crossfold needs.

Syntax

crossfold model [model_if] [model_in] [model_weight], 
        [eif()] [ein()] [eweight(varname)] 
        [stub(string)] [k(value)] [loud]
        [mae] [r2]
        [model_options]

suggest that the command you issue should be something like the following, where I replace a long string of independent variables with "..." to improve readability.

crossfold reghdfe ltrade lconflict_o ... wto_d year_*,  absorb(country_pair) k(299) r2"

Alternatively, if you need to use local macros for some other reason, your existing code could perhaps read

local model "reghdfe ltrade lconflict_o ... wto_d year_*, absorb(country_pair)"
local k 299
crossfold `model' k(`k') r2

The key is that your local macro model includes the comma separating the reghdfe command from its options, so a second comma would confuse the crossfold command.

[Kendal zhang]

Get rid of the quotes around 299. With quotes, you define k to contain the contents of local 299. This local does not exist, so the contents are empty, and k inherits that. When you evaluate k, the code default to 5 since there's nothing inside k.

Thank you, it works. How can I judge the result of k-fold?

[Kendal zhang]

Looking at the output of help crossfold, it seems likely that using macros, as you have done, will in any event not result in the syntax crossfold needs.

Syntax

crossfold model [model_if] [model_in] [model_weight],
[eif()] [ein()] [eweight(varname)]
[stub(string)] [k(value)] [loud]
[mae] [r2]
[model_options]

suggest that the command you issue should be something like the following, where I replace a long string of independent variables with "..." to improve readability.

crossfold reghdfe ltrade lconflict_o ... wto_d year_*, absorb(country_pair) k(299) r2"

Alternatively, if you need to use local macros for some other reason, your existing code could perhaps read

local model "reghdfe ltrade lconflict_o ... wto_d year_*, absorb(country_pair)"
local k 299
crossfold `model' k(`k') r2

The key is that your local macro model includes the comma separating the reghdfe command from its options, so a second comma would confuse the crossfold command.

local model "reghdfe ltrade lconflict_o ... wto_d year_*, absorb(country_pair)" local k 299 crossfold `model' k(`k') r2 when I use this code,it shows "command k is unrecognized". And the first code excecutes well.
besides, I want to have the result like this
Variable Obs Mean Std. Dev. Min Max

cv 130,364 5.204045 3.676834 .0000759 23.34942

but the code given by you can't realize this purpose.

[William Lisowski]

The problem reported in #6 is because at the time the crossfold command was run, the local macro model was undefined, so the command Stata tried to execute became

crossfold k(299) r2

or perhaps even

crossfold k() r2

if the local macro k was also undefined.

It is almost certain that this happened because you selected, in your do-file editor, the local commands and ran them, and then selected the crossfold command and ran it. The example below explains why that is a mistake.

I will also add, in case it wasn't apparent, that when I typed

local model "reghdfe ltrade lconflict_o ... wto_d year_*, absorb(country_pair)"

I replaced a number of variable names with ... to make it more easily readable on Statalist. That is not Stata syntax and you need to replace the ... with the omitted variable names.

Running a do-file with local macros

In the do-file editor window, I have a two-line program that I run in its entirety.

. do "/Users/lisowskiw/Downloads/example.do"

. local message Hello, world.

. display "The message is `message'"
The message is Hello, world.

. 
end of do-file

.

Now I run the same two lines by selecting the first line and running it, then selecting the second line and running it.

. do "/var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//SD04017.000000"

. local message Hello, world.

. 
end of do-file

. do "/var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//SD04017.000000"

. display "The message is `message'"
The message is 

. 
end of do-file

.

The important thing to keep in mind is that local macros vanish when the do-file within which they were created ends. If you look carefully at the results above, you'll see that when I selected a single line to run, it was copied into a temporary do-file and run, so even though both lines are in the same window in the do-file editor, they are run as separate do-files, and local macro defined in the first line vanishes at the end of that do-file, and is undefined when the second line is run.

So you defined the local macro model without running the crossfold command at the same time, and once that do-file ended model was no longer defined.