Trouble merging datasets with variables containing special (Portuguese) characters

[Daniel Franken]

I am using Stata 17, and I am having trouble merging between datasets based on variables containing Portuguese characters. It looks like both datasets are Unicode translated already (although one of them is much older). When I try to marge, variable values with special characters (such as â) don't get matched.

Another clue, if I tab the variable name, it appears as it should in the output. Example: "luziânia"
Yet in the log file it looks like this: "luzia^nia".

Can anyone point me in the right direction?

[George Ford]

https://www.stata.com/statalist/archive/2010-11/msg00100.html
https://www.statalist.org/forums/forum/general-stata-discussion/general/1413550-remove-special-characters-from-string

[Andrew Musau]

I usually run the following to remove such characters and standardize names:

* Example generated by -dataex-. To install: ssc install dataex
clear
input str10 name
"luziânia"
end

replace name = ustrto(ustrnormalize(name, "nfd"), "ascii", 2)

Res.:

. l
     +----------+
     |     name |
     |----------|
  1. | luziania |
     +----------+

.