QUESTION: .do file does not execute commands//prints commands in Results, does not produce .log or .dta [example .do and data included]

[Calaia Jackson]

Hi,
This is my first question on Statalist in all the years (10+) I have been using Stata so thanks for all of your contributions!

Issue: My .do file does not execute (e.g., producing a log file and saving new data) when I click "Do" (produces no output in the Results window). However, individual parts of the .do file execute when I select and "Do Selection" only (it runs, and stops with any errors). I have cleared all data, reopened Stata and opened the .do file expecting it to change the working directory, import or use saved data, and execute the commands, resulting in a .log file, and saved data in my working directory, but executing the .do file only produces lines of code preceded by '>' in the Results window, and no .log file or new data.

I am using version 17 of StataBE. I have created multiple .do files that start with a similar structure (see below) and include hundreds of lines of data cleaning commands (e.g., foreach loops, rename, label, tab, describe, etc.) but no regression commands or anything with estimations.

Any assistance (other than suggestions to upgrade to Stata 18 🙂) is appreciated. Thank you!

.do file content example snippet:

******/*/*/*/***Program Name: Data Clean PSES Teacher Survey 2022-23 ******/*/*/*/***
******/*/*/*/***Author: C. JACKSON******/*/*/*/***


**//uses the PSES_Teacher Open Data Survey for 2022-2023 retrieveed from SDP Open Data in March 2024********************************************** *******

version 17.0
clear all
capture log close
set linesize 80
set more off
set varabbrev off
display "$S_DATE"


*setting up directory and log
cd "PATH/Analysis Data"

log using "PSES_TeacherSurveyCleaning.log", replace


*Importing 2022-2023 PSES Teacher survey open data file--by school survey responses
import excel "PSES_Teacher_Open-Data_2022-2023.xlsx", sheet("by School") firstrow case(lower) allstring clear



browse foreach var of varlist _all {
cap assert `var'[4] == "Percent"if !_rc drop `var'
}foreach var of varlist a-g {
loc name = strlower(strtoname(`var'[4]))
cap rename `var' `name'

loc lbl = `name'[4]
la var `name' "`lbl'"
}

rename ulcs ulcscode

save "PSES_Teacher_Survey_2022_2023.dta", replace

foreach var of varlist q3a-df {
loc lbl = `var'[2]
la var `var' "How often - `lbl'"

loc name = strlower(strtoname(`var'[3]))
rename `var' `var'_`name'
}

*q3a
rename j_rarely q3a_rarely
rename l_occasionally q3a_occasionally
rename n_most_or_all_of_the_time q3a_most_all

save "clean_PSES_Teacher_Survey_2022_2023.dta", replace

log close




Data Example (Dataex) below:

[CODE]
* Example generated by -dataex-. For more info, type help dataex
clear
input str4 ulcscode str74 school_name str16 learning_network str11	sector	str14	response_count str17 response_rate	str13 met_threshold
"" ""		"" ""	"" ""	""
"" ""		"" ""	"" ""	""
"" ""		"" ""	"" ""	""
"ULCS" "School Name"		"Learning Network" "Sector"	"Response Count" "Response Rate"	"Met Threshold"
"1010" "Bartram, John High School"		"Network 13" "District"	"8" ".25"	"Yes"
"1020" "West Philadelphia High School"		"Network 13" "District"	"4" ".0952380952380952"	"No"
"1030" "High School of the Future"		"Innovation" "District"	"25" ".581395348837209"	"Yes"
"

Thanks!

[Andrew Musau]

but executing the .do file only produces lines of code preceded by '>' in the Results window, and no .log file or new data.

This indicates that the do file runs successfully. Since you have a capture command, any error message that terminates the execution of the do-file is being suppressed. There appear to be several errors in the code you provided, but it's unclear whether these errors are present in your actual do-file or if they are as a result of pasting the code here. To ensure clarity, please use CODE delimiters when pasting the do-file code. For example:

browse foreach var of varlist _all {
cap assert `var'[4] == "Percent"if !_rc drop `var'
}foreach var of varlist a-g {
loc name = strlower(strtoname(`var'[4]))
cap rename `var' `name'

should be:

browse
foreach var of varlist _all {
    assert `var'[4] == "Percent"
    if !_rc{
        drop `var'
    }
}
foreach var of varlist a-g {
   loc name = strlower(strtoname(`var'[4]))
   rename `var' `name'
}
save "PSES_Teacher_Survey_2022_2023.dta", replace

[Calaia Jackson]

Andrew-- The errors in the code example are absolutely from copy/paste, but I will use delimiters next time. Or perhaps just upload my .do file in it's entirety.

My question is not regarding this code. The code executes properly when I Execute Do for the selection, but the question is about executing the entire .do file. I don't think your response addresses my issue regarding not generating a .log file or new .dta file, though. Hopefully someone else can assist with the actual question posed.

[Andrew Musau]

You need to log the session to generate a log file.

help log

Others will likely need more information to assist you.

[Calaia Jackson]

Andrew--please read the post more closely.

There is a log using command in my code snippet.

Thanks for your responses.

[Calaia Jackson]

Reposting code snippet for readability:

******/*/*/*/***Program Name: Data Clean PSES Teacher Survey 2022-23 ******/*/*/*/***
******/*/*/*/***Author: C. JACKSON******/*/*/*/***


**//uses the PSES_Teacher Open Data Survey for 2022-2023 retrieveed from SDP Open Data in March 2024********************************************** *******

version 17.0
clear all
capture log close
set linesize 80
set more off
set varabbrev off
display "$S_DATE"


*setting up directory and log
cd "PATH/Analysis Data"

log using "PSES_TeacherSurveyCleaning.log", replace


*Importing 2022-2023 PSES Teacher survey open data file--by school survey responses
import excel "PSES_Teacher_Open-Data_2022-2023.xlsx", sheet("by School") firstrow case(lower) allstring clear


browse

foreach var of varlist _all {
cap assert `var'[4] == "Percent"if !_rc drop `var'
}

foreach var of varlist a-g {
loc name = strlower(strtoname(`var'[4]))
cap rename `var' `name'

loc lbl = `name'[4]
la var `name' "`lbl'"
}

rename ulcs ulcscode

save "PSES_Teacher_Survey_2022_2023.dta", replace

foreach var of varlist q3a-df {
loc lbl = `var'[2]
la var `var' "How often - `lbl'"

loc name = strlower(strtoname(`var'[3]))
rename `var' `var'_`name'
}

*q3a
rename j_rarely q3a_rarely
rename l_occasionally q3a_occasionally
rename n_most_or_all_of_the_time q3a_most_all

save "clean_PSES_Teacher_Survey_2022_2023.dta", replace

log close

[Mike Lacy]

I'd echo Andrew's point about the use of -cap-. If I were trying to diagnose your problem, I'd start with the old-fashioned strategy of inserting a bunch of -display- commands, e.g. -display "Reached 1"- -display "Reached 2" - etc. throughout the code so as to be sure I knew exactly where the failure occurred.

A stylistic suggestion of possible help: I'd avoid using /* or /* for ordinary comments, and just use * or // instead. It's too easy to accidentally create a sequence of /* and */ that comments out a block of code. I don't know that that's a problem here, but I've spent long minutes (hours?) trying to find a mistake like that myself.

[Andrew Musau]

Andrew--please read the post more closely.

There is a log using command in my code snippet.

Thanks for your responses.

Got it. As I said in #2, if #6 is your literal code, capture suppresses the error message and the do-file terminates without having closed the log or saved the file. Consider:

* Example generated by -dataex-. For more info, type help dataex
clear
input str1 var1 str7 var2 str1 var3
"a" ""        "b"
"a" ""        "b"
"a" ""        "b"
"a" "Percent" "b"
"p" ""        "b"
"a" ""        "b"
end

foreach var of varlist _all {
    assert `var'[4] == "Percent"if !_rc drop `var'
}

Res.:

. foreach var of varlist _all {
  2. 
. cap assert `var'[4] == "Percent"if !_rc drop `var'
  3. 
. }

. foreach var of varlist _all {
  2. 
.  assert `var'[4] == "Percent"if !_rc drop `var'
  3. 
. }
invalid 'drop' 
r(198);

Try deleting capture and see what happens.

[Devra Golbe]

Calaia,

I posted the code in #6 into the editor and tried to run it. Here's the beginning of the output:

. ******/*/*/*/***Program Name: Data Clean PSES Teacher Survey 2022-23 ******/*/*/*/***
> ******/*/*/*/***Author: C. JACKSON******/*/*/*/***
>
>
> **//uses the PSES_Teacher Open Data Survey for 2022-2023 retrieveed from SDP Open Data in March 2024*******
> *************************************** *******
>
> version 17.0
> clear all
> capture log close

As Andrew noted, this indicates that your do file DOES execute, but Stata seems to think it's all one line. As Mike suggested, your erroneous commenting of the first few lines is causing trouble. Try removing everything before the version statement. Here's what I see when I do that and run the code. It quits with the expected error that it can't find your directory on my machine, but it's running, and it should create a log file as requested.

. version 17.0

. clear all

. capture log close

. set linesize 80

. set more off

. set varabbrev off

. display "$S_DATE"
27 Apr 2024

.
.
. *setting up directory and log
. cd "PATH/Analysis Data"
unable to change to PATH/Analysis Data
r(170);

[Calaia Jackson]

Mike, Andrew, Devra--thank you all so much for your helpful suggestions.

Both removing the capture command and changing the comments at the top helped to resolve my issue and the .do file executed and generated the new data and log file as expected.

I appreciate your help!