Matching superscript in string variable

[paulvonhippel]

The attached example dataset contains 14 rows of a single string variable called Year. In most rows, Year just contains the 4-digit year (e.g., "1992") but in a few rows there is a superscript (e.g., "1992¹").

I would like to flag the rows with a superscript. Here is how I try to do that:

use example, clear
gen superscript = (substr(Year,-1,1) == "¹")
list

But this doesn't work. In every row the value of superscript is 0.
What's a better way to do this?

Thanks!
Paul

[paulvonhippel]

Here's another symptom of the same problem. To avoid attaching a dataset, I first tried to create an example with code.

input str5 Year
1992
1994¹
end
list

But the listed data looked weird in my output window. Instead of displaying the superscript 1, Stata displayed a "?" in a box, which I suspect means that it didn't recognize or couldn't display the superscript, even though I see superscripts in the example data that I attached to my previous post.

If I understood how Stata represents superscripts in string variables, that would probably resolve both problems.

Thanks again!

[Girish Venkataraman]

Regex might resolve it by searching out all non-numbers (is how Stata sees those superscripts) I think you need a newer version of Stata for ustrregexra functions (perhaps Stata 16+) to be available.

replace Year = ustrregexra(Year, "\D+", "")

or

replace  Year = substr(Year, 1, 4)

[paulvonhippel]

Thanks! Your code replaces the Year variable. How would I generate a superscript variable that indicates whether a superscript is present in Year?

[Bjarte Aagnes]

gen has_superscript_one = usubstr(Year,-1,1) == "¹"

. di ustrlen("¹")
1

. di strlen("¹")
2

or more general

gen has_superscript = ustrregexm(Year,"^\d{4}\p{No}+$")

[Girish Venkataraman]

Thanks! Your code replaces the Year variable. How would I generate a superscript variable that indicates whether a superscript is present in Year?

Either of these will do it. If the position and numbers of strange characters remain the same.

gen supyes = strlen(Year)!=4

gen supyes = !missing(substr(Year, 5, .))

[paulvonhippel]

My collaborator Yujin Kwon reports that this works too:

gen superscript = strpos(Year,"¹")>0

I like her solution because it looks explicitly for the superscript rather than counting on the string length, which might vary for other reasons....

[Girish Venkataraman]

My collaborator Yujin Kwon reports that this works too:

gen superscript = strpos(Year,"¹")>0

I like her solution because it looks explicitly for the superscript rather than counting on the string length, which might vary for other reasons....

..presuming the superscript text is always 1, it will work. The choice of solution depends on the data at hand.