Macro to Create several monthly .dta files

[Mustafa Zia]

I have several .Dat files and each file is for a seperate month and seperate year. The years range is 2010 to 2012, and the months are Jan-Dec for 2010, Jan-Dec for 2011, and Jan-Apr for 2012. Again, I have a seperate .DAT file for each month of each year. I can obviously edit the program separately to run for each month of each year and create a monthly .dta files for each, however, I wanted to create a macro that would accomplish this without having to change the .do files every time I run it for a different .dat file.

So for example, the following top part creates a .dta file for the month of January 2010.

local dat_name "C:\Users\ziam\Desktop\CPS\jan10pub.dat"
/* The following line should contain the path to your output '.dta' file */

local dta_name "C:\Users\ziam\Desktop\CPS\DtaFiles\cpsb2010_1 .dta "


/* The following line should contain the path to the data dictionary file */

local dct_name "I:\CPS Data\.dct files\cpsbjan10.dct"

/* The line below does NOT need to be changed */

quietly infile using "`dct_name'", using("`dat_name'") clear

And this works perfectly fine. However, I used the following code in hoping of creating a separte .dta file for each month of year without having to change the code over and over again.

The Macro Program (the part in red is where i have the macro code)


set mem 500m



local mons `=strlower(c(Mons))' // lowercase months (Jan -> jan)

forvalues year = 10(1)12 {
foreach month of local mons {

local m : list posof "`month'" in mons // map jan to 1, feb to 2, ...

capture confirm file "C:\Users\ziam\Desktop\CPS\DtaFiles\`year'`mon th'p ub.dta" // check that raw file exists

/* Clean if file exits exists */
if !_rc {
local dat_name "C:\Users\ziam\Desktop\CPS\`month'`year'pub.da t"
local dta_name "C:\Users\ziam\Desktop\CPS\DtaFiles\cpsb20`yea r'_` m'.dta"
local dct_name "I:\CPS Data\.dct files\cpsb`month'`year'.dct"
quietly infile using "`dct_name'", using("`dat_name'") clear
/*------------------------------------------------

All items, except those with one character, also can have values
of -1, -2, or -3 even if the values are not in the documentation
The meaning is
-1 .Blank or not in universe
-2 .Don't know
-3 .Refused

The following changes in variable names have been made, if necessary:
'$' to 'd'; '-' to '_'; '%' to 'p';
($ = unedited data; - = edited data; % = allocated data)

Decimal places have been made explict in the dictionary file.
Stata resolves a missing value of -1 / # of decimal places as a missing value.
-----------------------------------------------*/

** These note statements incorporate variable universes into the Stata data file.
note: by Jean Roth, [email protected] Mon Feb 28 13:28:35 EST 2011
note hrmonth: U ALL HHLD's IN SAMPLE
note hryear4: U ALL HHLDs IN SAMPLE
note hurespli: U ALL HHLDs IN SAMPLE
note hufinal: U ALL HHLDs IN SAMPLE
note huspnish: U ALL HHLDs IN SAMPLE
note hetenure: U ALL HHLDs IN SAMPLE
note hehousut: U HRINTSTA = 1 OR HUTYPB = 1-3
note hetelhhd: U ALL HHLDs IN SAMPLE
note hetelavl: U HRINTSTA = 1
note hephoneo: U HETELHHD = 2
note hefaminc: U HETELHHD = 1 OR HETELAVL = 1
note hutypea: U HETELHHD = 1 OR HETELAVL = 1
note hutypb: U HETELHHD = 1 OR HETELAVL = 1
note hutypc: U HETELHHD = 1 OR HETELAVL = 1
note hwhhwgt: U HETELHHD = 1 OR HETELAVL = 1
note hrintsta: U HRINTSTA = 1
note hrnumhou: U ALL HHLDs IN SAMPLE
note hrhtype: U ALL HHLDs IN SAMPLE
note hrmis: U ALL HHLDs IN SAMPLE
note huinttyp: U ALL HHLDs IN SAMPLE
note huprscnt: U ALL HHLDs IN SAMPLE
note hrlonglk: U ALL HHLDs IN SAMPLE
note hrhhid2: U ALL HHLDs IN SAMPLE
note hubus: U ALL HHLD's IN SAMPLE
note hubusl1: U ALL HHLD's IN SAMPLE
note hubusl2: U ALL HHLD's IN SAMPLE
note hubusl3: U ALL HHLD's IN SAMPLE
note hubusl4: U ALL HHLD's IN SAMPLE
note gereg: U ALL HHLD's IN SAMPLE
note gestcen: U ALL HHLD's IN SAMPLE
note gestfips: U ALL HHLD's IN SAMPLE
note gtcbsa: U ALL HHLD's IN SAMPLE
note gtco: U ALL HHLD's IN SAMPLE
note gtcbsast: U ALL HHLD's IN SAMPLE
note gtmetsta: U ALL HHLD's IN SAMPLE
note gtindvpc: U ALL HHLD's IN SAMPLE
note gtcbsasz: U ALL HHLD's IN SAMPLE
note gtcsa: U ALL HHLD's IN SAMPLE
note proldrrp: U ALL HHLD's IN SAMPLE
note pupelig: U PRPERTYP = 1, 2, OR 3
note perrp: U PRPERTYP = 1, 2, OR 3
note peparent: U PRPERTYP = 1, 2, OR 3
note peage: U EVERY PERSON
note prtfage: U PRPERTYP = 1, 2, 0R 3
note pemaritl: U PRPERTYP = 1, 2, 0R 3
note pespouse: U PEAGE >= 15
note pesex: U PEMARITL = 1
note peafever: U PRPERTYP = 1, 2, 0R 3
note peafnow: U PEAGE >=17
note peeduca: U PRPERTYP = 2 or 3
note ptdtrace: U PRPERTYP = 2 0R 3
note prdthsp: U PRPERTYP = 1, 2, 0R 3
note puchinhh: U PEHSPNON = 1
note pulineno: U PEHSPNON = 1
note prfamnum: U PEHSPNON = 1
note prfamrel: U PRPERTYP = 1, 2, 0R 3
note prfamtyp: U PRPERTYP = 1, 2, 0R 3
note pehspnon: U PRPERTYP = 1, 2, 0R 3
note prmarsta: U PRPERTYP = 1, 2, 0R 3
note prpertyp: U PRPERTYP = 2 0R 3
note penatvty: U ALL HOUSEHOLD MEMBERS
note pemntvty: U PRPERTYP = 1, 2, 0R 3
note pefntvty: U PRPERTYP = 1, 2, 0R 3
note prcitshp: U PRPERTYP = 1, 2, 0R 3
note prcitflg: U PRPERTYP = 1, 2, 0R 3
note prinusyr: U PRPERTYP = 1, 2, 0R 3
note puslfprx: U PRCITSHP = 2, 3, 4, OR 5
note pemlr: U PRCITSHP = 2, 3, 4, OR 5
note puwk: U PRPERTYP = 2
note pubus1: U PRPERTYP = 2
note pubus2ot: U PRPERTYP = 2
note pubusck1: U PRPERTYP = 2
note pubusck2: U PRPERTYP = 2
note pubusck3: U PRPERTYP = 2
note pubusck4: U PRPERTYP = 2
note puretot: U PRPERTYP = 2
note pudis: U PRPERTYP = 2
note peret1: U PRPERTYP = 2
note pudis1: U PEMLR = 5 AND (PURETOT = 1 OR
note pudis2: U PEMLR = 5 AND (PURETOT = 1 OR
note puabsot: U PEMLR = 5 AND (PURETOT = 1 OR
note pulay: U PEMLR = 5 AND (PURETOT = 1 OR
note peabsrsn: U PEMLR = 5 AND (PURETOT = 1 OR
note peabspdo: U PEMLR = 2
note pemjot: U PEABSRSN = 4-12, 14
note pemjnum: U PEMLR = 1, 2
note pehrusl1: U PEMJOT = 1
note pehrusl2: U PEMJOT = 1 OR 2 AND PEMLR = 1 OR 2
note pehrftpt: U PEMJOT = 1 AND PEMLR = 1 OR 2
note pehruslt: U PEHRUSL1 = -4 OR PEHRUSL2 = -4
note pehrwant: U PEMLR = 1 OR 2
note pehrrsn1: U PEMLR = 1 AND
note pehrrsn2: U PEHRWANT = 1 (PEMLR = 1 AND PEHRUSLT < 35)

*Everything below this point, aside from the final save, are value labels

#delimit ;

;
label values hufinal hufinal;
label define hufinal
1 "FULLY COMPLETE CATI INTERVIEW"
2 "PARTIALLY COMPLETED CATI"

........and so on until I close the loop below

compress
saveold `dta_name' , replace

}

/* Display message if file does not exist */
else {
di "No data for `month' in 20`year'"
}

}
}


However, my program doesn't create the .dta files and the do file just keeps running: What am I doing wrong? Am I closing the loop incorrectly?

The following log displays while my program continues to run:

. /* Display message if file does not exist
> */
. else {
586. di "No data for `month' in 20`year'
> "
587. }
588.
. }
589. }
No data for jan in 2010
No data for feb in 2010
No data for mar in 2010
No data for apr in 2010
No data for may in 2010
No data for jun in 2010
No data for jul in 2010
No data for aug in 2010
No data for sep in 2010
No data for oct in 2010
No data for nov in 2010
No data for dec in 2010
No data for jan in 2011
No data for feb in 2011
No data for mar in 2011
No data for apr in 2011
No data for may in 2011
No data for jun in 2011
No data for jul in 2011
No data for aug in 2011
No data for sep in 2011
No data for oct in 2011
No data for nov in 2011
No data for dec in 2011
No data for jan in 2012
No data for feb in 2012
No data for mar in 2012
No data for apr in 2012

[William Lisowski]

Welcome to Statalist!

Your problem lies in this command:

capture confirm file "C:\Users\ziam\Desktop\CPS\DtaFiles\`year'`mon th'p ub.dta"

Consider the following:

. local year 10

. local month Jan

. display "C:\Users\ziam\Desktop\CPS\DtaFiles\`year'`month'pub.dta"
C:\Users\ziam\Desktop\CPS\DtaFiles`year'Janpub.dta

. display "C:/Users/ziam/Desktop/CPS/DtaFiles/`year'`month'pub.dta"
C:/Users/ziam/Desktop/CPS/DtaFiles/10Janpub.dta

.

You see that in the first display command, the sixth \ prevents the ` following it from being interpreted as the opening of a macro reference. Changing \ to / solves the problem, and despite the fact that you are running on Windows, Stata will interpret the path correctly. The advice I have seen on Statalist generally recommends always using / instead of \ in paths to avoid this problem.

In many (most? all?) cases, the \ shields any special meaning for the following character. I'm sorry I cannot easily find a Stata help file that explains this better; perhaps someone else can point to an appropriate reference.

[Clyde Schechter]

Well, from the output Stata is giving you, it is following the part of the code that applies when the data file with the specified name is not found. Assuming that what you have posted is exactly the code you ran, the reason for this is that:

capture confirm file "C:\Users\ziam\Desktop\CPS\DtaFiles\`year'`mon th'p ub.dta" // check that raw file exists

contains a blank space in the middle of `mon th'--which should be `month'. Since there is no defined local macro `mon th', this evaluates to an empty string, so that you are looking for filenames like 12p ub.dta. I also suspect that the blank between p and ub in the same line is a mistake. After you fix that, you can at least get into the main part of the code. But then you will stumble again when you reach

local dta_name "C:\Users\ziam\Desktop\CPS\DtaFiles\cpsb20`yea r'_` m'.dta"

because, once again, there is an extraneous blank space between yea and r, and yet another between ` and m. I haven't read down the rest of the code. It seems that you are getting a lot of spurious blank spaces generally. I suggest you carefully proof read the entire thing and eliminate all of those.

By the way, a matter of terminology. The term "macro" in Stata refers to local and global macros, which are strings used to store information for use in programming. You are using the term "macro" the way it is used in SAS, to refer to a program--we call those programs.

Added after seeing William Lisowski's post: William has identified a different problem that escaped my notice, but he is also correct. You will need to fix both of these problems.

[Nick Cox]

On what is (is not) a macro in Stata see also http://www.stata.com/statalist/archi.../msg01258.html

On backslashes that stab you:

See 18.3.11 in http://www.stata.com/manuals14/u18.pdf

and/or

SJ-8-3 pr0042 . . . . . . . Stata tip 65: Beware the backstabbing backslash
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. J. Cox
Q3/08 SJ 8(3):446--447 (no commands)
tip to use forward slash instead of backslash for
directory and file names

Go directly there: http://www.stata-journal.com/sjpdf.h...iclenum=pr0042

[Nick Cox]

Cross-posting of http://stackoverflow.com/questions/3...-read-all-data

Please note our policy of cross-posting, which is that you should tell us about it.

[Mustafa Zia]

Cross-posting of http://stackoverflow.com/questions/3...-read-all-data

Please note our policy of cross-posting, which is that you should tell us about it.

I sincerely apologize. As a new member, was not aware of that rule.

Also, thank you to everyone who helped. The backward slash turned out to be my issue.

[Nick Cox]

Thanks for the apology, but all members are asked to read the FAQ Advice before posting.