PSA: Replacing string values does not entirely remove the old values from the saved .dta file

[Nils Enevoldsen]

I want to alert people who de-identify, anonymize, or share data to a behavior of Stata's .dta writer. I reported it to StataCorp Technical Support on 7 September. Details of their reply are at the end.

In short: replace name = "" does not remove names from a saved dataset. Neither does replacing names with shorter pseudonyms. The old values stay in the file, minus their first byte. Stata does not show them, but anyone can read them with a text editor or grep.

1. String data

When a str# value is replaced by a shorter one, Stata writes the new value and a null terminator, and leaves the rest of the fixed-width field as it was. The rest of the old value is then saved to disk. Here is a reproduction on StataNow 19.5:

version 19
clear
input str30 id
"Maria Fernanda Oliveira"
"João Carlos Pereira"
"Ana Beatriz Santos"
end
replace id = string(_n)
save pseudonymised.dta, replace

Stata shows id as 1, 2 and 3. On disk, the three 30-byte fields are

1\0ria Fernanda Oliveira\0\0\0\0\0\0\0
2\0ão Carlos Pereira\0\0\0\0\0\0\0\0\0\0
3\0a Beatriz Santos\0\0\0\0\0\0\0\0\0\0\0\0

Blanking the names with replace id = "" gives the same result. So does -saveold- to formats 117 and 115. -help dta- documents this at the level of the file format: a short string is followed by "random bytes". But those bytes are not random. They are the value you just overwrote.

Dropping the variable removes its contents from that save. However, freed string memory gets reused. Any string variable you create later in the same session, including one in a new dataset after -clear-, can pick up the dropped values and write them into the next file you save.

I scanned a few hundred published datasets from public research archives. About a quarter of those with a str# variable wider than some of its values have readable text in the padding that appears in no visible value in the file. The text includes addresses, place names and free-text responses. In one file, the padding held the full names of several individuals who appear nowhere in the dataset's visible values. The files most at risk are those with wide string variables that hold short values.

2. Metadata

The fixed-width metadata slots (variable names, formats, value label names, variable labels, sort list) are also written without being cleared first. The padding contains whatever memory Stata last used for them. That can include names and labels from a different dataset saved earlier in the same session. Using -clear-, -clear all- or -label drop _all- in between does not help. The padding also survives a load and re-save in a fresh session. About half of the published files I scanned have fragments like this. You can see it in the auto dataset Stata ships with: the slot for -turn- contains "turn\0ng_circle".

mata: fh = fopen(findfile("auto.dta"), "r"); s = fread(fh, 100000); fclose(fh); strpos(s, "ng_circle")

StataCorp's response

Technical Support told me that StataCorp does not consider this a bug, but they said it is prompting internal discussion. One option they are considering is zeroing the unused space in strings on -save-, either automatically or through an option, because doing it on every save could slow Stata down.

In my view, the current behavior is a disclosure risk for personally identifiable information (PII).

[daniel klein]

Thank you for bringing this to my attention!

I agree that this behavior definitely poses a disclosure risk, particularly because the current documentation seems misleading.

I was able to replicate the issue with fixed-width strings (not sure about strLs). I could not, however, find similar issues with variable names, variable labels, or other metadata.

The obvious workaround for fixed-width string variables appears to be issuing a compress command before saving the file to disk. However, having to remember to address the problem in one way or another in every session or do-file isn't exactly convenient. I'd strongly prefer zeroing out the unused bytes on save to be the default behavior, rather than an option. If performance really is an issue, then perhaps an additional c() setting that users can turn on and off, including permanently, might be the best approach.

[Nils Enevoldsen]

Thanks, Daniel. Two reproducers below, both tested on StataNow 19.5.

Metadata. The easiest way to see it is to rename a variable or value label. The name slots are fixed width, and the new name is written over the old one without clearing the rest of the slot. The second part shows that the leftover bytes carry into an unrelated dataset, even after -clear all-.

version 19

* 1. Rename a variable and its value label, then save
clear
set obs 1
gen byte nda_partner_acmebank = 1
label define terms_loan_acmebank 1 "yes"
label values nda_partner_acmebank terms_loan_acmebank
rename nda_partner_acmebank q17
label copy terms_loan_acmebank yesno
label values q17 yesno
label drop terms_loan_acmebank
describe
save meta1.dta, replace

* 2. Build an unrelated dataset after clear all, then save
clear all
set obs 1
gen byte x = 1
save meta2.dta, replace

* Non-zero positions mean the old names are in the file
mata:
fh = fopen("meta1.dta", "r"); s = fread(fh, 1e6); fclose(fh)
strpos(s, "partner_acmebank"), strpos(s, "loan_acmebank")
fh = fopen("meta2.dta", "r"); s = fread(fh, 1e6); fclose(fh)
strpos(s, "partner_acmebank"), strpos(s, "loan_acmebank")
end

Both files contain both old names. In meta1.dta, the variable name slot reads q17\0partner_acmebank. meta2.dta has a single variable, x, and no value labels, but it contains the same fragments.

compress. -compress- only helps when it changes the storage type. If the longest value still fills the width, the type stays the same and the old values stay in the file:

version 19
clear
set obs 4
gen party = ""
replace party = "Movimento Democrático pela Reconstrução Nacional" in 1
replace party = "União Progressista das Cooperativas Agrícolas" in 2
replace party = "Frente Popular Independente" in 3
replace party = "Respondent declined to say which party they voted for" in 4

replace party = "A" in 1
replace party = "B" in 2
replace party = "C" in 3
compress party
save parties.dta, replace

* Non-zero positions mean the old values are in the file
mata:
fh = fopen("parties.dta", "r"); s = fread(fh, 1e6); fclose(fh)
strpos(s, "Reconstrução Nacional"), strpos(s, "Cooperativas Agrícolas"), strpos(s, "Popular Independente")
end

-compress- reports 0 bytes saved, party stays str53, and all three original party names are in the file.

strLs. strLs should be fine, except in the rare case when the strL values happen to be the exactly the working memory that is "randomly" in the file. That memory can be anything that belongs to Stata. For example __icd10pcs.dta, shipped in ado/base, happens to contain a few lines of source code from merge.ado.

[daniel klein]

compress. -compress- only helps when it changes the storage type. If the longest value still fills the width, the type stays the same and the old values stay in the file:

Very important catch, thanks again! I totally missed that, because we'd typically replace all observations with fixed-length strings such as "" or "anonymized". I hadn't considered the case where compress does not actually change the storage type.