[Chul-Kyoo Jung]

William Lisowski , Sorry for the unclear description for the codes, which I should have shown in #1. The codes consists of two stages:

1) OLS regression for the estimates for all firms' expected outputs (lny_10) and error terms (epsilon) in each industry so that
2) In the 2nd stage (MATA), using GMM, I estimates for all output production function coefficients of each firm in each industry by relying on a simple assumption that productivity follows a random process.

So throughout these two processes, I am expecting to generate "DLW simplified ind `i'.dta" where `i' denotes each industry.
But like you explained, the local macro is no longer defined outside the loop, I guess that's why I cannot have all the .dta for each industry.

I think now the remaining question is how to modify the remaining part of the code after executing stage 1 and 2 so that I may be able to produce each .dta file for each industry.

Let me try again based on your suggestions and others. Please let me know if you have any tip for the improvement. Any suggestion would be greatly appreciated.

Thank you so much.


p.s. For now, the most updated version of the original codes based on all the suggestions above is the following:

qui {
clear mata

foreach i of numlist 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
use if ksic1c == `i'  using "D:\markupest\firmdata.d3.dta", clear
noisily display "KSIC `i' `c(N)' observations"
// ksic1c: 2-digit KSIC
*----------------------------------------------------------------------------------------------------*
* OLS REGRESSION FOR STARTING VALUES & First stage
// xi: reg y l k i.year
xi: reg lny_10 lnl lnk_10 i.year                        
// lny_10: log of Real Value Added (denominated by 2010 PPI), lnl: log of Number of Workers, lnk_10: log of Real Average Capital Stock (denominated by 2010 PPI)
gen blols=_b[lnl]
gen bkols=_b[lnk_10]
predict phiS
predict epsilon, res
label var phiS "Expected value added"
label var epsilon "Measurement error first stage"

bys firmid (year): gen phiS_lag=phiS[_n-1]                // firmid: company ID
bys firmid (year): gen lnl_lag=lnl[_n-1]
bys firmid (year): gen lnk_10_lag=lnk_10[_n-1]
*---COMPUTE CORRECTED SHARES-------------------------------------------------------------------------*
gen va_c=exp(lny_10-epsilon)
gen alpha_l=wb_10/va_c                                // wb: Real Wage Bill (denominated by 2010 PPI)
*---------------------------------------------------------------------------------------------------*
drop _I*
sort firmid year
gen const=1
drop if lnl_lag==.
drop if lnk_10==.
drop if phiS==.
drop if phiS_lag==.
drop if lny_10==.
}

qui {
*-------------------------------------BEGIN MATA PROGRAM--------------------------------------------*
mata:
void GMM_DLW(todo,betas,crit,g,H)
{
    PHI=st_data(.,("phiS"))
    PHI_LAG=st_data(.,("phiS_lag"))
    Z=st_data(.,("const","lnl_lag","lnk_10"))
    X=st_data(.,("const","lnl","lnk_10"))
    X_lag=st_data(.,("const","lnl_lag","lnk_10_lag"))
    Y=st_data(.,("lny_10"))
    C=st_data(.,("const"))

    OMEGA=PHI-X*betas'
    OMEGA_lag=PHI_LAG-X_lag*betas'
    OMEGA_lag_pol=(C,OMEGA_lag)
    g_b = invsym(OMEGA_lag_pol'OMEGA_lag_pol)*OMEGA_lag_pol'OMEGA
    XI=OMEGA-OMEGA_lag_pol*g_b
    crit=(Z'XI)'(Z'XI)
}

void DLW()
    {
S=optimize_init()
optimize_init_evaluator(S, &GMM_DLW())
optimize_init_evaluatortype(S,"d0")
optimize_init_technique(S, "nm")
optimize_init_nmsimplexdeltas(S, 0.1)
optimize_init_which(S,"min")
optimize_init_params(S,(0,0,0))
p=optimize(S)
p
st_matrix("beta_dlw",p)
}
end
*-----------------------------------------END MATA PROGRAM---------------------------------------*
cap program drop dlw
program dlw, rclass
preserve
sort firmid year
mata DLW()
end

*------------------------------------COMPUTE MARKUPS --------------------------------------------*
// OLS, DLW estimates for variable input (here labor) are used to compute markup distribution

*------------------------------------ACF estimates-----------------------------------------------*
dlw
gen beta_cS=beta_dlw[1,1]
gen beta_lS=beta_dlw[1,2]
gen beta_kS=beta_dlw[1,3]
gen mu_dlwS=beta_lS/alpha_l                            // Markup
gen tfp_dlwS=phiS-beta_lS*lnl-beta_kS*lnk_10                // TFP
*-------------------------------------------------------------------------------------------------*

* collect markup estimates and productivity for analysis:
keep firmid year ksic1c mu_dlwS tfp_dlwS lbg30 lny lny_10 lnl lnk_10 beta_*S

save "DLW simplified ind `i'.dta"
}
}


*** Appending the data sets
use "DLW simplified ind 3.dta", clear
save "DLW simplified total.dta", replace

foreach I of numlist 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
    use "DLW simplified total.dta", clear
    append using "DLW simplified ind `I'.dta"
    save "DLW simplified total.dta", replace
}

[Bjarte Aagnes]

I think now the remaining question is how to modify the remaining part of the code after executing stage 1 and 2 so that I may be able to produce each .dta file for each industry.

One long file is hard to understand, maintain and debug. If you separate parts of the code, possibly in a "master" do file the structure will be clear. First, the program definitions can be in a separate do file. Next, if two steps are done within the same loop (data subset) , each step could be in a separate file. The code below is just one suggestion (and not tested), there may be parts of your problem I do not understand.

* START master do file

version 17

run program_defs.do 

local firmdata D:\markupest\firmdata.d3.dta
local industries 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91 

foreach i of numlist `industries' {

    use if ( ksic1c ==`i' ) using "`firmdata'", clear

    do step_1.do 
    do step_2.do

    save "DLW simplified ind `i'.dta"
}

clear

foreach i of numlist `industries' {

    append using "DLW simplified ind `i'.dta"    
}

save "DLW simplified total.dta"

* END master do file

[Chul-Kyoo Jung]

Bjarte Aagnes , thank you so much. When I follow your suggestion,
I encountered the following error, which seems to occur when the code runs step_2.do.
I was wondering if this is because the foreach loop generates the same matrices in the mata.
I would appreciate if anyone could give me a tip to fix this problem.

. mata:
------------------------------------------------- mata (type end to exit) -----------
: void GMM_DLW(todo,betas,crit,g,H)
GMM_DLW() already exists
(30 lines skipped)
-------------------------------------------------------------------------------------
r(3000);

step_2.do is the following:

qui {
*-------------------------------------BEGIN MATA PROGRAM--------------------------------------------*
mata:
void GMM_DLW(todo,betas,crit,g,H)
{
    PHI=st_data(.,("phiS"))
    PHI_LAG=st_data(.,("phiS_lag"))
    Z=st_data(.,("const","lnl_lag","lnk_10"))
    X=st_data(.,("const","lnl","lnk_10"))
    X_lag=st_data(.,("const","lnl_lag","lnk_10_lag"))
    Y=st_data(.,("lny_10"))
    C=st_data(.,("const"))

    OMEGA=PHI-X*betas'
    OMEGA_lag=PHI_LAG-X_lag*betas'
    OMEGA_lag_pol=(C,OMEGA_lag)
    g_b = invsym(OMEGA_lag_pol'OMEGA_lag_pol)*OMEGA_lag_pol'OMEGA
    XI=OMEGA-OMEGA_lag_pol*g_b
    crit=(Z'XI)'(Z'XI)
}

void DLW()
    {
S=optimize_init()
optimize_init_evaluator(S, &GMM_DLW())
optimize_init_evaluatortype(S,"d0")
optimize_init_technique(S, "nm")
optimize_init_nmsimplexdeltas(S, 0.1)
optimize_init_which(S,"min")
optimize_init_params(S,(0,0,0))
p=optimize(S)
p
st_matrix("beta_dlw",p)
}
end
*-----------------------------------------END MATA PROGRAM---------------------------------------*
cap program drop dlw
program dlw, rclass
preserve 
sort firmid year
mata DLW()
end

*------------------------------------COMPUTE MARKUPS --------------------------------------------*
// OLS, DLW estimates for variable input (here labor) are used to compute markup distribution

*------------------------------------ACF estimates-----------------------------------------------*
dlw
gen beta_cS=beta_dlw[1,1]
gen beta_lS=beta_dlw[1,2]
gen beta_kS=beta_dlw[1,3]
gen mu_dlwS=beta_lS/alpha_l                            // Markup
gen tfp_dlwS=phiS-beta_lS*lnl-beta_kS*lnk_10                // TFP
*-------------------------------------------------------------------------------------------------*

* collect markup estimates and productivity for analysis:
keep firmid year ksic1c mu_dlwS tfp_dlwS lbg30 lny lny_10 lnl lnk_10 beta_*S

[William Lisowski]

You need to place the

clear mata

into program_defs.do, which is what the code you describe as step2.do corresponds to. These program definitions must be before the loop, not within it.

[Bjarte Aagnes]

First, the program definitions can be in a separate do file.

The definitions of the mata functions and the dlw program should be in do file program_defs.do which is run one time at start. At the top of this file before going to mata add the Stata commands:

capture mata : mata drop GMM_DLW() 
capture mata : mata drop DLW()

[Chul-Kyoo Jung]

William Lisowski , Bjarte Aagnes Thank you so much for all your helps. I really appreciate all your patience and kindness.