Bug? Unbalanced curly braces in strings cause Stata to crash

[Matthew Hall]

While testing my shell wrapper inshell I believe I have come across a bug in Stata (I'm running Stata/SE 17.0 for Mac (Intel 64-bit), Revision 13 Oct 2022). I had written a program to generate random strings using ASCII character codes 32 to 126 and kept the ones containing dollars signs ($), backticks (`), and double quotes ("), then tested each one to see if inshell could echo (and subsequently capture) it. Some examples are presented below:

$jR'2a;RD8VdEM\peyskw't^
n2R$+eaHQkvtNAo-CseTU/0g
^=wuRQgsdeRaMJjh$-3on05^
wup5@C':Pm=ndcDkH=o="+at
u4.~.A~weH-gcI(ln`w08;Wi
,~^G<A@Z6+#0$`qb[*|m,daU
Op<|U`GF4;hZLFmf5BI%4!a~
LC"3,=#y3zwj^U`\W_HC%~)N
$#qApyEN4hG@C#AML}QW7ZD~
$e.wv]p^}/txI,d+>Sq<XIXr

After about 5000 successful tests with similar strings I found that certain of these would crash Stata, requiring a force quit. Upon further inspection, I noticed that all of these strings had one thing in common: unbalanced curly braces ({}). Take for example these two strings:

j1W|671}O/|Deo{?$$E*s}R8
}CW{?`_)AFp}gJPbgCkdPj_9

Though the string can be stored it cannot be displayed. In my particular case, if the string is contained in an external file which inshell reads , it will crash when attempting to store that string as a returned macro. However, any Stata user should be able to reproduce the issue with the following code:

WARNING! the following code is likely to crash your copy of Stata and possibly your machine:

local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"
macro list _test_string
display "`macval(test_string)'"   // this will crash Stata
display "`test_string'" _asis     // this will also crash Stata

This will happen even if the string is stored as a scalar. String parsing functions will still work, such as:

display strpos("`test_string'", "A")
9

Can anyone else reproduce this? I'm assuming that Stata is attempting to display what it thinks are SMCL directives, however the _asis option to display does not change this.

[Hemanshu Kumar]

Yes, I'm on Stata/MP 17 for Mac, and your string causes Stata to become non-responsive for me. I need to "Force Quit" it. As you said, string functions do work on it. But displaying it causes a crash. Very strange.

[William Lisowski]

I can reproduce your results on my copy of Stata/SE 17.0 for Mac (Apple Silicon) Revision 13 Oct 2022 running on my MacBook Air (M2, 2022) still running macOS Monterey 12.6.1.

I'm unsurprised by anything the display command does, it's why I try to remember to use macro list when displaying macro contents here on Statalist.

I would bring your example to the the attention of Stata Technical Services at https://www.stata.com/support/tech-support/

Here is another work-around.

local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"
macro list _test_string
frame create to_print
frame to_print: set obs 1
frame to_print: generate str50 test = "`macval(test_string)'"
frame to_print: list test, noheader noobs

. local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"

. macro list _test_string
_test_string:   }CW{?`_)AFp}gJPbgCkdPj_9

. frame create to_print

. frame to_print: set obs 1
Number of observations (_N) was 0, now 1.

. frame to_print: generate str50 test = "`macval(test_string)'"

. frame to_print: list test, noheader noobs
  +--------------------------+
  | }CW{?`_)AFp}gJPbgCkdPj_9 |
  +--------------------------+

.

[Hua Peng (StataCorp)]

I can reproduce it on Stata 16 MP on Windows. We will look into it.

[Matthew Hall]

I can reproduce your results on my copy of Stata/SE 17.0 for Mac (Apple Silicon) Revision 13 Oct 2022 running on my MacBook Air (M2, 2022) still running macOS Monterey 12.6.1.

I'm unsurprised by anything the display command does, it's why I try to remember to use macro list when displaying macro contents here on Statalist.

I would bring your example to the the attention of Stata Technical Services at https://www.stata.com/support/tech-support/

Here is another work-around.

local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"
macro list _test_string
frame create to_print
frame to_print: set obs 1
frame to_print: generate str50 test = "`macval(test_string)'"
frame to_print: list test, noheader noobs

. local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"

. macro list _test_string
_test_string: }CW{?`_)AFp}gJPbgCkdPj_9

. frame create to_print

. frame to_print: set obs 1
Number of observations (_N) was 0, now 1.

. frame to_print: generate str50 test = "`macval(test_string)'"

. frame to_print: list test, noheader noobs
+--------------------------+
| }CW{?`_)AFp}gJPbgCkdPj_9 |
+--------------------------+

.

So it must be in the display command, because this happens (relying on the fact that display will print the first observation of the variable test):

local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"
macro list _test_string
frame create to_print
frame to_print: set obs 1
frame to_print: generate str50 test = "`macval(test_string)'"
frame to_print: list
frame to_print: display test     // crashes Stata

[Hua Peng (StataCorp)]

The fix will take soime time to make the update, a temporary work around is using:

 
 display _asis "`test_string'"

Note that _asis must be in front of the string in order to avoid triggering the bug.

[Matthew Hall]

Hemanshu, William, and Hua— Thanks for looking into this. I did not initially discover this by using the display command, rather the crash happened while testing my shell wrapper inshell. I was already looking for nefarious strings which I thought might cause syntax errors, rather than crashes, and I could not isolate anything by viewing the program trace log because inshell would crash (before Stata was able to print the trace statement) immediately after the shell line. But upon further testing it seems the issue is with the Mata printf() and display()functions (though not sprintf()), and it seems to be with SMCL:

local test_string "}CW{?`_)AFp}gJPbgCkdPj_9"
macro list _test_string
display _asis "`test_string'"
mata :
A = st_local("test_string")
A
sprintf(A)
display(A, 1)        // (s, 1) does not interpret SMCL directives
//    printf(A)      // this will crash Stata
//    display(A)     // this will also crash Stata
end

It's certainly an edge case, but thanks again to everyone for checking this out.

Shameless plug: some of you might find my shell wrapper inshell useful. It has been thoroughly tested on Unix-based OSes (macOS and Linux) with as many shells as I could run (including Microsoft PowerShell) and is compatible with Microsoft Window as well. You can download it from the SSC with ssc install inshell

[Hua Peng (StataCorp)]

Thanks for reporting it and the example to help us identify the bug. We have the fix and it will in a future Stata 17 update, not likely in the next update but the one after the next update.