precombine merge error

[Emory Davis]

I am using -precombine- to check a set of files before appending them, and getting an error that I can't figure out how to troubleshoot. (I am using version 17.)

local dta_path "C:/Users/.../Data/Working Data/Academic/Reading"

local files: dir "`dta_path'" files "final*.dta"

cd `"`dta_path'"' 

precombine `files', clear uniquevars describe(type)

I get this error: after merge, not all observations matched (merged result left in memory). In the "Reports relevant to the combining of the following datasets" output prior to this error, it shows that they all have the same number of variables. There are different numbers of observations, but that shouldn't be an issue since I'm not actually trying to merge the data. There is no data in memory after this error and I don't know how I can access the "merged result left in memory" to try to figure out what is going on. I have manually checked each one of the dta files and cannot figure out why this is happening. I can append the files without issue and get exactly what I expect. Even though I am only appending a small number of files (~10) in this case, I want to be able to use -precombine- as a best practice. I ran -precombine- successfully on an earlier version of these files and never got an error. How can I trouble shoot this? What is causing the error?

[Eric Makela]

Hi Emory, so aside from reviewing the changes you made to the data cleaning code, I'd suggest that only you can tell what's causing the error, since you have not posted a reproducible example. Additionally, what's preventing you from simply documenting the number of observations in each dataset, then appending them all into one and seeing if the total adds up?

User-written programs such as 'precombine' are consistently updated and tested, but are not guaranteed against error. Here are some suggestions if you are keen on pre-authenticating and cleaning the data:

1) are the variable types the same in each file, and are the variable names the same? (even capitalization can trigger unwanted errors)
2) if (1) is not correct, then why don't you write a loop program that opens each file, renames the variables to a standard, changes them to a desired type, then saves the file?
3) only (1) will cause issues when appending files

[Emory Davis]

Hi Eric Makela, unfortunately it's slightly tricky for me to post a reproducible example because the data contains identifiable student information and I would have to alter all of the data to de-identify it. In response to your suggestions: The dta files are created using a loop that imports the data from Excel files and standardizes the variable types. I have verified that the variable names are the same in all files. I am able to append the files and get the expected number of observations, so there is nothing in the data that seems to prevent appending successfully. Hence why I'm confused about why -precombine- is throwing an error and I'm stumped on how to troubleshoot it.

[Eric Makela]

No worries, then! -trace- is Stata's primary debug command, and its help file will let you know how to find the source of errors in future projects.

[Michael Duarte Goncalves]

Eric Makela, Good morning,

Could you give me more insights about your post #2, the point 2) please?

In fact, I would like to create a loop that open each dta file and rename each variable in order to have homogeneous variable labels, for example.

Thank you so much for your help.

Best,

Michael