Combining Frames After Cleaning

[Mark Weiss]

I'm new to Stata and I'm not sure what the normal process is for importing, manipulating, and combining multiple files.

I have Census data from 2018-2021 that shows income and household size. There are extraneous rows in each excel file, so when I import them, I want to keep only the necessary rows and then combine the files into one single frame. On my first attempt, after cleaning the first file, I wrote

frame create census19
frame change census19

import excel *pathway to next file*

and then cleaned the next file. However, when I write

append using census 18

where census18 is the file I imported first, I get an error message.

Am I going about things incorrectly? I know that one can link frames together, but is it possible to do this, clean a frame, and then combine observations from both? Apologies in advance if my terminology is vague or misused, still getting used to everything!

[Clyde Schechter]

You can link frames and then do the equivalent of -merge- by using -frget- or the -frval()- function. But for appending, there is nothing in official Stata. There is, however, three user written commands available from SSC that do this: Jeremey Freese' -frameappend-, Jurgen Wiemers' -fframeappend-, and Roger Newson's -xframeappend-. -frameappend- was the first out of the gate and is the one I use for this purpose. There may be additional useful features in the others, or faster performance, or the like--I don't know.

[Hemanshu Kumar]

Flagging this since you say you're very new to Stata: you may not need frames at all. Import the first file, process it, save it as a Stata dataset, work on the next one, save that, and so on.. and then append them all together. Maybe put this in a loop if the processing is identical for each file.

Something like this:

local file1 "pathway_to_first_file"
local file2 "pathway_to_second_file"
local file3 "pathway_to_third_file"

forvalues i=1/3 {
   import excel using `"`file`i''"', clear
   ...
   tempfile file_`i'
   save `file_`i''
}

clear
append `file_1' `file_2' `file_3'