Syntax error r(198) when trying to loop an rdrobust command in a matrix

[Andreas Esbjornsen]

Dear stata forum,

I am writing a paper on the causal effect of compulsory school reforms, expanding on this study (Cavaille & Marshall 2019). Inspired by their do-file, I wish to produce many regression tables/matrices at once (loop) with both multiple x's (the reforms) and multiple y's (which cohorts were affected). I can make it work when only looping the y's, but I can not make the double loop work. I get syntax error r(198) when using the following code:

local j=1
quietly foreach x in "DK (1972)" "FR (1967)" "GB (1947)" "GB (1972)" "NL (1974)" "NL (1942)" "SE (1965)" "BE (1983)" "DE-E (1959)" "IT (1963)" "GR (1975)" "PT (1956)" {
    local i=0
    matrix ALL_FS_`j' = J(6,9,0)
    matrix rownames ALL_FS_`j' = "Reform" "SE" "p-value" "Bandwidth" "Observations" "Outcome mean"
    matrix colnames ALL_FS_`j' = "(1)" "(2)" "(3)" "(4)" "(5)" "(6)" "(7)" "(8)" "(9)"
    noisily di "******************** Reform = `x' **********************"
    foreach y in eduyrs plus12 plus13 plus14 plus15 plus16 plus17 plus18 tertiary {
        local i=`i'+1
        rdrobust `y' running if reform=="`x'", c(0)
        matrix ALL_FS_`j'[1,`i'] = e(tau_cl)
        matrix ALL_FS_`j'[2,`i'] = e(se_cl)
        matrix ALL_FS_`j'[3,`i'] = e(pv_cl)
        matrix ALL_FS_`j'[4,`i'] = floor(e(h_bw))
        matrix ALL_FS_`j'[5,`i'] = e(N_l) + e(N_r)
        sum `y' if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
        matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        if `i'==1 {
            sum eduyrs if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
            matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        }
    }
    noisily estout matrix(ALL_FS_`j', fmt("3 3 4 0 0 2")), style(tex)
    local j=`j'+1
}

I am quite inexperienced when it comes to loops and matrices. Can you help me?

I am running Stata 16.1 on Mac

Best regards,
Andreas Esbjørnsen
Department of Political Science
University of Copenhagen

[William Lisowski]

Perhaps if you try

local j=1
local x "DK (1972)"
    local i=0
    ...
    noisily estout matrix(ALL_FS_`j', fmt("3 3 4 0 0 2")), style(tex)
    local j=`j'+1

Stata will reveal the source of your syntax error.

The most common source of the problem you describe would be submitting the code you show in post #1 in sections. If you run the first two lines, and then run the later code, the values of local macros x and j will be undefined in the later code, because local macros vanish when the do-file within which they were created ends.

[Andreas Esbjornsen]

Hi William

Thanks for the advice. I now get the error message:

c() should be set within the range of running
r(125);

However, my c is 0, which is within the range of running (from roughly -50 to 50).

I have even tried another running variable, but I get the same error.

Best regards,
Andreas

[William Lisowski]

my c is 0, which is within the range of running (from roughly -50 to 50)

A confident assertion, but no output is shown to support it. What matters in this case is that the range of running when reform=="DK (1972)" contains 0.

I suggest doing

tabstat running, statistics(min max) by(reform)

to confirm your assertion.

[Andreas Esbjornsen]

Thank you so much again for replying, William.

I have attached a screenshot of my output when using the command:

tabstat running, statistics(min max) by(reform)

As you can see, c() is within the range of running, and (with the exception of a few early and late reforms) running is actually quite well distributed around 0.

I have been searching the internet and its fora quite extensively, but I can't find anyone who are having similar problems. Can you help me?


My best,
Andreas

[Andreas Esbjornsen]

I am, however, able to execute following commands:

foreach x in "DK (1972)" "FR (1967)" "GB (1947)" "GB (1972)" "NL (1974)" "NL (1942)" "SE (1965)" "BE (1983)" "DE-E (1959)" "IT (1963)" "GR (1975)" "PT (1956)" {
  rdrobust eduyrs running if reform=="`x'"
 }

As well as

foreach y in eduyrs plus12 plus13 plus14 plus15 plus16 plus17 plus18 tertiary {
    rdrobust `y' running if reform=="DK (1972)"
}

Which means that the problem arises in the loop or matrix configurations.

[William Lisowski]

First of all, you may find today's post #5 at the following rdrobust topic to be informative

https://www.statalist.org/forums/for...22#post1583122

as well as the topic at

https://www.statalist.org/forums/for...egress-command

Secondly, be sure you have the latest rdrobust per post #9 in this other topic.

https://www.statalist.org/forums/for...64#post1581564

and if it turns out you don't, update it and again try the code you ran for post #3.

Thirdly, building on your post #6, where does the following fail:

foreach x in "DK (1972)" "FR (1967)" "GB (1947)" "GB (1972)" "NL (1974)" "NL (1942)" "SE (1965)" "BE (1983)" "DE-E (1959)" "IT (1963)" "GR (1975)" "PT (1956)" {
foreach y in eduyrs plus12 plus13 plus14 plus15 plus16 plus17 plus18 tertiary {
    display "reform `x' dependent variable `y'"
    quietly rdrobust `y' running if reform=="`x'", c(0)
}
}

Try this code both with and without the c(0) option.

[Andreas Esbjornsen]

Again, William, thank you so much for taking your time.

I was able to run the code successfully both with and without the c(0) option. So the problem must be with the matrix / regression table.

[William Lisowski]

I fail to believe anything you are doing outside the rdrobust command could cause rdrobust to generate an error message.

I have been assuming that the error reported in post #3 (a) occurred on the first execution of the rdrobust command and (b) your do-file stopped immediately after reporting the error message. Is that the case?

I also assume that prior to post #8 you confirmed that you are indeed running the latest version of rdrobust as discussed in post #7. Is that the case?

Looking back at the original code in post #1 I find I fail to understand the rationale behind the if block.

...
        sum `y' if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
        matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        if `i'==1 {
            sum eduyrs if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
            matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        }
...

When `i'==1 then `y' will be eduyrs, so the second pair of commands should do nothing different than the first pair. I note that this block does not appear in the original authors' code that you adapted. Why did you choose to add it?

[Andreas Esbjornsen]

Dear William (and thanks again. You are truly saving me here!),

First of all, I think I might have misunderstood you in post #2, because I tried to code with all the reforms "DK (1972)" "FR (1967)"... etc. But I now see that you meant for me to only try with "DK (1972)".
Correcting my mistake and doing as you told me, the code looks like this:

local j=1
local x "DK (1972)"
    local i=0
    matrix ALL_FS_`j' = J(6,9,0)
    matrix rownames ALL_FS_`j' = "Reform" "SE" "p-value" "Bandwidth" "Observations" "Outcome mean"
    matrix colnames ALL_FS_`j' = "(1)" "(2)" "(3)" "(4)" "(5)" "(6)" "(7)" "(8)" "(9)"
    noisily di "******************** Reform = `x' **********************"
    foreach y in alder_afsluttet plus12 plus13 plus14 plus15 plus16 plus17 plus18 tertiær {
        local i=`i'+1
        rdrobust `y' running if reform=="`x'", c(0)
        matrix ALL_FS_`j'[1,`i'] = e(tau_cl)
        matrix ALL_FS_`j'[2,`i'] = e(se_cl)
        matrix ALL_FS_`j'[3,`i'] = e(pv_cl)
        matrix ALL_FS_`j'[4,`i'] = floor(e(h_bw))
        matrix ALL_FS_`j'[5,`i'] = e(N_l) + e(N_r)
        sum `y' if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
        matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        if `i'==1 {
            sum alder_afsluttet if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
            matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        }
    }
    noisily estout matrix(ALL_FS_`j', fmt("3 3 4 0 0 2")), style(tex)
    local j=`j'+1

And it actually looks much better now. It successfully produces the first seven rdrobust-regressions as well as the matrix. The only problem appears to be the plus18 variable and tertiary variable...

[Andreas Esbjornsen]

It works!! (Almost...)

I can't explain why, but apparently all the values of plus18 was == 0. When I fixed that, I was able to run the original code from post #1 successfully!

However, when I realized the nature of my mistake, I found out that I had to test from plus11 to plus17 instead of plus12 to plus18 (all are dummy variables indicating the age or more, when you finished school).

Curiously enough, I am now able to run the command with plus12 to plus18 flawlessly, but when I run it with plus11 to plus17, I arrive at a syntax error at the Swedish reform (1965):

local j=1
quietly foreach x in "DK (1972)" "FR (1967)" "GB (1947)" "GB (1972)" "NL (1974)" "NL (1942)" "SE (1965)" "BE (1983)" "DE-E (1959)" "IT (1963)" "GR (1975)" "PT (1956)" {
    local i=0
    matrix ALL_FS_`j' = J(6,9,0)
    matrix rownames ALL_FS_`j' = "Reform" "SE" "p-value" "Bandwidth" "Observations" "Outcome mean"
    matrix colnames ALL_FS_`j' = "(1)" "(2)" "(3)" "(4)" "(5)" "(6)" "(7)" "(8)" "(9)"
    noisily di "******************** Reform = `x' **********************"
    foreach y in eduyrs plus11 plus12 plus13 plus14 plus15 plus16 plus17 tertiary {
        local i=`i'+1
        rdrobust `y' running if reform=="`x'", c(0)
        matrix ALL_FS_`j'[1,`i'] = e(tau_cl)
        matrix ALL_FS_`j'[2,`i'] = e(se_cl)
        matrix ALL_FS_`j'[3,`i'] = e(pv_cl)
        matrix ALL_FS_`j'[4,`i'] = floor(e(h_bw))
        matrix ALL_FS_`j'[5,`i'] = e(N_l) + e(N_r)
        sum `y' if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
        matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        if `i'==1 {
            sum eduyrsif reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
            matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        }
    }
    noisily estout matrix(ALL_FS_`j', fmt("3 3 4 0 0 2")), style(tex)
    local j=`j'+1
}

Output:

[Andreas Esbjornsen]

I have found out the mistake, although I do not understand it. Likewise with plus18 all being==0, when plus11==1 (because people are older when they finish school) it shows as a syntax error. I do not think it has anything to do with the syntax, but it does not matter. I found out the mistake, and I am able to overcome it manually.

Thank you so much for your time and help, William. I would have spent weeks on this issue without your valuable knowledge. I am very grateful!

Best regards,
Andreas

[William Lisowski]

The syntax error likely comes from not from your code but from within rdrobust Your anomalous data presented it with a situation for which it was not prepared and for which it had done no checking to avoid, and so - most likely - some local variable contained a value (or no value) which when Stata substituted it into the code produced a syntax error, rather than a more suitable diagnostic error message. So while your understanding of your data was insufficient, the rdrobust code was also insufficiently robust to unsuitable data.

[William Lisowski]

Let me add further analysis. Your code in post #11 is

local j=1
quietly foreach x in "DK (1972)" "FR (1967)" "GB (1947)" "GB (1972)" "NL (1974)" "NL (1942)" "SE (1965)" "BE (1983)" "DE-E (1959)" "IT (1963)" "GR (1975)" "PT (1956)" {
    local i=0
    matrix ALL_FS_`j' = J(6,9,0)
    matrix rownames ALL_FS_`j' = "Reform" "SE" "p-value" "Bandwidth" "Observations" "Outcome mean"
    matrix colnames ALL_FS_`j' = "(1)" "(2)" "(3)" "(4)" "(5)" "(6)" "(7)" "(8)" "(9)"
    noisily di "******************** Reform = `x' **********************"
    foreach y in eduyrs plus11 plus12 plus13 plus14 plus15 plus16 plus17 tertiary {
        local i=`i'+1
        rdrobust `y' running if reform=="`x'", c(0)
        matrix ALL_FS_`j'[1,`i'] = e(tau_cl)
        matrix ALL_FS_`j'[2,`i'] = e(se_cl)
        matrix ALL_FS_`j'[3,`i'] = e(pv_cl)
        matrix ALL_FS_`j'[4,`i'] = floor(e(h_bw))
        matrix ALL_FS_`j'[5,`i'] = e(N_l) + e(N_r)
        sum `y' if reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
        matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        if `i'==1 {
            sum eduyrsif reform=="`x'" & running>=-floor(e(h_bw)) & running<=floor(e(h_bw))
            matrix ALL_FS_`j'[6,`i'] = `r(mean)'
        }
    }
    noisily estout matrix(ALL_FS_`j', fmt("3 3 4 0 0 2")), style(tex)
    local j=`j'+1
}

I previously mentioned that I do not understand what the code highlighted in red accomplishes that wasn't already accomplished by the previous sum command and would omit it. Note that this version has a typographical error with no space before the "if" in the sum command. That should throw a "variable not found" error rather than a syntax error, however, so I don't think it was the source of the mysterious problem.

Another problem is that you have wrapped all your work in quietly to produce attractive output. But then you cannot see what Stata is doing as it goes, and this can make hard to locate the approximate position of the error.

In general the use of quietly when you are having problems is not a good idea. The quietly prefix should be reserved for code that is working. If previously working code now fails after changes are made, removing quietly is the first step in debugging.

[Andreas Esbjornsen]

You are absolutely right about everything. I had merely adapted the code in red from Cavaille & Marshall but you are right, I don't need it and it works fine without.

I will be aware of 'quietly' in the future. Thank you very much, William!