How do I use regular expressions to match a certain number of occurrences of a given expression?

[Michael Anbar]

I have data like this:

clear

input str14 var1
"a_123456"
"b_456789"
"777888_c"
"12345_d_000111"
end

and I want to match the first occurrence of a six-digit number, with missing otherwise, in Stata 13.1. I can do this:

gen num = regexs(1) if regexm(var1, "([0-9][0-9][0-9][0-9][0-9][0-9])")

but is there a way to abbreviate this? Many programming languages and software packages allow for syntax like this:

gen num = regexs(1) if regexm(var1, "([0-9]){6}")

to match exactly six occurrences, or something like this, to match anywhere from 3-6 occurrences:

gen num = regexs(1) if regexm(var1, "([0-9]){3,6}")

or

gen num = regexs(1) if regexm(var1, "([0-9])\{3,6\}")

The POSIX standard allows this, and Stata's documentation claims that its syntax is "nearly identical" to this standard. Is this supported in Stata?

Thank you,
Michael Anbar

[Nick Cox]

I think the short answer is No.

The best documentation of what is allowed that I know is http://www.stata.com/support/faqs/da...r-expressions/

[Michael Anbar]

That's unfortunate. Maybe the documentation for -regexm()- and the other regex functions is simply incorrect, at least to my reading, because that documentation, unlike the FAQ you linked to, certainly implies that Stata supports the POSIX standard. Maybe improving this functionality beyond the basic syntax (although I would argue that searching for simple repetitions *is* basic) will happen in the next version. Thank you for the help.

[Phil Schumm]

I can do this:

gen num = regexs(1) if regexm(var1, "([0-9][0-9][0-9][0-9][0-9][0-9])")

but is there a way to abbreviate this?

The notation {m,n} (matching the preceding element at least m and not more than n times) is not currently supported by Stata's implementation of regular expressions. Hopefully, it will be added at some point. Until then, one possible workaround is

regexm(var1, `"(`="[0-9]"*6')"')

This is admittedly a bit ugly, and not that much help in this particular case, but in cases where you are looking for a larger number of consecutive instances, it can be helpful.

[Nick Cox]

To be clear: Phil's neat trick here doesn't mean that regexm() supports any special syntax for repeated elements. He's merely instructing Stata to evaluate a string expression before the result is passed to regexm().

[Michael Anbar]

Thank you for the help; I'm familiar with Stata's syntax for macro expressions, but I agree that it's definitely a kludge, albeit one that only works for matching a number of occurrences exactly, not the {m, n} syntax (as Phil mentioned).

[Nick Cox]

Cross-posted at

http://stackoverflow.com/questions/2...rrences-of-a-g

Please note our policy on cross-posting, which is that you should tell us about it. This is explicit in the FAQ Advice.

[Keith Finlay]

I find Stata's regex functions so limiting that I often use the shell command to access something more powerful, like grep. Probably absurdly slow. grep in Stata 14, please?