Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Mobility poverty using Synthetic panel data

    I am doing a poverty mobility analysis based on the paper "Measuring Poverty Dynamics with Synthetic Panels Based on Repeated Cross-Sections" (https://doi.org/10.1111/obes.12539).
    I am having trouble finding the value of ρ (using Proposition 2- p.10 method). My data is appended from two sets of surveys in two different time periods, and the households surveyed in the two time periods may not be the same.
    Here is a part of the code to calculate ρ that I wrote myself, it works well, but I am honestly worried about its accuracy.
    gen yob = year - age (yob: year of birth)
    local y1 = 2020
    local y2 = 2022
    local age_min = 25
    local age_max = 55
    local diff = `y2' - `y1'
    local age_min_p2 = `age_min' + `diff'
    local age_max_p2 = `age_max' + `diff'
    gen filter = 0
    replace filter = 1 if year == `y1' & (age >= `age_min' & age <= `age_max')
    replace filter = 1 if year == `y2' & (age >= `age_min_p2' & age <= `age_max_p2')
    keep if filter == 1
    drop filter
    egen panid = group(yob sex edu urban province)
    label var panid
    xtreg ln_thubq, fe i(panid)

  • #2
    hi, can anyone help me, please?!

    Comment


    • #3
      You define your cohorts (I guess panid) and collapse the Y and X to cohort. Then regress Y on X.

      Comment


      • #4
        If you are doing synthetic panel analyses, have you checked this link including the manual and the do-file?

        Comment


        • #5
          Great find, Seungmin Lee, but note that the do-file text at that link states, at one part,
          " ** Option to estimate rho based on cohorts has been removed. The only remaining option is to import an income correlation coefficient from an ancillary panel" FWIW, LEE CHAN, like the code author, I think the weakest link in the Dang & Lanjouw methodology is the point estimate part (relying on a point estimate of "rho" via cohorts), and I would avoid it as far as possible. How narrow/wide are the bounds on the transition probability estimates?

          Comment


          • #6
            Thanks for the reminder Prof Stephen Jenkins . I read the above research paper, it seems that the author, although suggesting that from ancillary data is allowed, also gave an alternative method to calculate Rho using Proposition 2- p.606, and I am following that method, using the above self-written commands, but I am worried about its accuracy.

            Comment


            • #7
              This is what I see in Prop 2. the subscript changes from i to c, so the sample is collapsed to cohort defined by as many variables as you choose. (I can't test this, but it will be something like it).
              Code:
              egen panid = group(yob sex edu urban province)
              label var panid
              preserve
                   collapse (mean) yob sex edu urban province thubq , by(panelid)
                   g ln_thubq = ln(thubq)
                   reg ln_thubq yob sex edu urban province  
              restore

              Comment


              • #8
                This might be useful: https://povertyevidence.org/wp-conte...ynpan_tool.txt

                Comment


                • #9
                  Maybe this will help.

                  I took if from: https://www.rohanbyanjankar.com.np/2...ution-for.html

                  and recoded in Stata. Results are comparable (won't be exact due to bootstrap). The SE aren't showing, but I'll try to fix that later.

                  round data at:
                  https://github.com/rohanbjkr/synthet...ain/round1.dta
                  https://github.com/rohanbjkr/synthet...ain/round2.dta

                  Code:
                  *********************************************
                  * Synthetic Panel Tool - Point Estimates
                  * FIXED: Proper individual-level correlation
                  *********************************************
                  
                  capture program drop synpanel_point
                  program define synpanel_point, rclass
                      version 14.0
                      syntax , ///
                          df_round1(string) ///          
                          df_round2(string) ///          
                          x_cols(string) ///            
                          cohort_cols(string) ///        
                          dep_var_round1(string) ///    
                          dep_var_round2(string) ///    
                          povline_r1(string) ///          
                          povline_r2(string) ///          
                          [LOGtransform ///              
                          weight_var(string) ///        
                          n_bootstrap(integer 100) ///    
                          seed(integer 42) ///            
                          SAVEresults(string)]            
                      
                      **** STEP 1: Load and prepare data ****
                      
                      // Load round 1 data
                      use "`df_round1'", clear
                      
                      // Check that x_cols exist in round 1
                      foreach var of local x_cols {
                          confirm variable `var'
                      }
                      confirm variable `dep_var_round1'
                      confirm variable `povline_r1'
                      
                      // Check weight variable if specified
                      if "`weight_var'" != "" {
                          confirm variable `weight_var'
                      }
                      
                      gen byte round = 1
                      
                      // Get poverty line value from data
                      sum `povline_r1', meanonly
                      scalar pov_line1 = r(mean)
                      di "Poverty line round 1: " pov_line1
                      
                      tempfile temp1
                      save `temp1'
                      
                      // Load round 2 data
                      use "`df_round2'", clear
                      
                      // Check that x_cols exist in round 2
                      foreach var of local x_cols {
                          confirm variable `var'
                      }
                      confirm variable `dep_var_round2'
                      confirm variable `povline_r2'
                      
                      // Check weight variable if specified
                      if "`weight_var'" != "" {
                          confirm variable `weight_var'
                      }
                      
                      gen byte round = 2
                      
                      // Get poverty line value from data
                      sum `povline_r2', meanonly
                      scalar pov_line2 = r(mean)
                      di "Poverty line round 2: " pov_line2
                      
                      tempfile temp2
                      save `temp2'
                      
                      // Append datasets
                      use `temp1', clear
                      append using `temp2'
                      
                      // Create cohort identifier from cohort_cols
                      di ""
                      di "Creating cohorts from: `cohort_cols'"
                      egen cohort = group(`cohort_cols')
                      
                      // Check cohort sizes in round 2
                      di ""
                      di "Checking cohort sizes in round 2..."
                      
                      gen cohort_n = .
                      bysort cohort round: replace cohort_n = _N
                      
                      preserve
                      keep if round == 2
                      bysort cohort: keep if _n == 1
                      count if cohort_n < 10
                      local n_small = r(N)
                      
                      if `n_small' > 0 {
                          di as error "WARNING: `n_small' cohorts have fewer than 10 observations in round 2"
                          list cohort cohort_n if cohort_n < 10 in 1/10, noobs
                      }
                      restore
                      
                      drop cohort_n
                      
                      // Generate dependent variables
                      if "`logtransform'" != "" {
                          di ""
                          di "Log-transforming dependent variables..."
                          
                          // Drop if they already exist
                          capture drop log_dep_r1
                          capture drop log_dep_r2
                          
                          gen log_dep_r1 = ln(`dep_var_round1') if round == 1
                          gen log_dep_r2 = ln(`dep_var_round2') if round == 2
                          local depvar1 "log_dep_r1"
                          local depvar2 "log_dep_r2"
                          
                          // IMPORTANT: Log-transform poverty lines too!
                          scalar pov_line1 = ln(pov_line1)
                          scalar pov_line2 = ln(pov_line2)
                          di "Log poverty line round 1: " pov_line1
                          di "Log poverty line round 2: " pov_line2
                      }
                      else {
                          local depvar1 "`dep_var_round1'"
                          local depvar2 "`dep_var_round2'"
                      }
                      
                      **** STEP 2: Run regressions for each round ****
                      di ""
                      di "Running regression for round 1..."
                      
                      if "`weight_var'" != "" {
                          reg `depvar1' `x_cols' [pw=`weight_var'] if round == 1
                      }
                      else {
                          reg `depvar1' `x_cols' if round == 1
                      }
                      
                      predict res1 if e(sample), residuals
                      predict fitted1_for_r2 if round == 2, xb
                      
                      if "`weight_var'" != "" {
                          sum res1 if round == 1 [aw=`weight_var']
                      }
                      else {
                          sum res1 if round == 1
                      }
                      scalar sd_res1 = r(sd)
                      
                      di ""
                      di "Running regression for round 2..."
                      
                      if "`weight_var'" != "" {
                          reg `depvar2' `x_cols' [pw=`weight_var'] if round == 2
                      }
                      else {
                          reg `depvar2' `x_cols' if round == 2
                      }
                      
                      predict res2 if e(sample), residuals
                      predict fitted2_for_r2 if round == 2, xb
                      
                      if "`weight_var'" != "" {
                          sum res2 if round == 2 [aw=`weight_var']
                      }
                      else {
                          sum res2 if round == 2
                      }
                      scalar sd_res2 = r(sd)
                      
                      **** STEP 3: Estimate cohort-level correlations ****
                      di ""
                      di "Estimating cohort-level correlations..."
                      
                      // Create cohort mean of round 1 residuals
                      preserve
                      keep if round == 1 & !missing(res1)
                      collapse (mean) res1_cohort_mean=res1, by(cohort)
                      tempfile res1_means
                      save `res1_means'
                      restore
                      
                      // Merge cohort means onto round 2 observations
                      preserve
                      keep if round == 2 & !missing(res2)
                      merge m:1 cohort using `res1_means', keep(3) nogen
                      
                      // Now correlate at individual level
                      corr res1_cohort_mean res2
                      scalar rho_c = r(rho)
                      restore
                      
                      scalar rho_partial = rho_c  
                      
                      di "Cohort correlation: " %6.4f rho_c
                      di "Partial correlation: " %6.4f rho_partial
                      
                      // Save working dataset before bootstrap
                      tempfile working_data
                      save `working_data', replace
                      
                      **** STEP 4: Bootstrap loop ****
                      di ""
                      di "Starting bootstrap with `n_bootstrap' iterations..."
                      
                      tempname boot_results
                      matrix `boot_results' = J(`n_bootstrap', 6, .)
                      matrix colnames `boot_results' = rho_c rho_partial P11 P10 P01 P00
                      
                      set seed `seed'
                      
                      forvalues b = 1/`n_bootstrap' {
                          
                          if mod(`b', 10) == 0 | `b' == 1 {
                              di "Bootstrap iteration `b' of `n_bootstrap'"
                          }
                          
                          // Reload fresh data for each bootstrap iteration
                          use `working_data', clear
                          
                          // Bootstrap sample
                          bsample
                          
                          // Re-run regressions
                          qui reg `depvar1' `x_cols' if round == 1
                          qui predict res1_bs if e(sample), residuals
                          qui predict fitted1_bs if round == 2, xb
                          qui sum res1_bs if round == 1
                          scalar sd_res1_bs = r(sd)
                          
                          qui reg `depvar2' `x_cols' if round == 2
                          qui predict res2_bs if e(sample), residuals
                          qui predict fitted2_bs if round == 2, xb
                          qui sum res2_bs if round == 2
                          scalar sd_res2_bs = r(sd)
                          
                          // Calculate correlation at individual level
                          preserve
                          keep if round == 1 & !missing(res1_bs)
                          collapse (mean) res1_bs_cohort_mean=res1_bs, by(cohort)
                          tempfile res1_bs_means
                          save `res1_bs_means'
                          restore
                          
                          preserve
                          keep if round == 2 & !missing(res2_bs)
                          merge m:1 cohort using `res1_bs_means', keep(3) nogen
                          
                          qui corr res1_bs_cohort_mean res2_bs
                          scalar rho_c_bs = r(rho)
                          restore
                          
                          scalar rho_hat_bs = max(rho_c_bs, 0.01)  
                          
                          // Calculate transition probabilities using bivariate normal
                          tempvar z1 z2 P11 P10 P01 P00
                          
                          gen `z1' = (pov_line1 - fitted1_bs) / sd_res1_bs if round == 2
                          gen `z2' = (pov_line2 - fitted2_bs) / sd_res2_bs if round == 2
                          
                          // Poor-Poor (P11): Stayed poor
                          gen `P11' = binormal(`z1', `z2', rho_hat_bs) if round == 2
                          
                          // Poor-Nonpoor (P10): Escaped poverty
                          gen `P10' = binormal(`z1', -`z2', -rho_hat_bs) if round == 2
                          
                          // Nonpoor-Poor (P01): Fell into poverty
                          gen `P01' = binormal(-`z1', `z2', -rho_hat_bs) if round == 2
                          
                          // Nonpoor-Nonpoor (P00): Stayed non-poor
                          gen `P00' = binormal(-`z1', -`z2', rho_hat_bs) if round == 2
                          
                          // Store weighted means
                          if "`weight_var'" != "" {
                              qui sum `P11' [aw=`weight_var'] if round == 2
                              matrix `boot_results'[`b',3] = r(mean)
                              qui sum `P10' [aw=`weight_var'] if round == 2
                              matrix `boot_results'[`b',4] = r(mean)
                              qui sum `P01' [aw=`weight_var'] if round == 2
                              matrix `boot_results'[`b',5] = r(mean)
                              qui sum `P00' [aw=`weight_var'] if round == 2
                              matrix `boot_results'[`b',6] = r(mean)
                          }
                          else {
                              qui sum `P11' if round == 2
                              matrix `boot_results'[`b',3] = r(mean)
                              qui sum `P10' if round == 2
                              matrix `boot_results'[`b',4] = r(mean)
                              qui sum `P01' if round == 2
                              matrix `boot_results'[`b',5] = r(mean)
                              qui sum `P00' if round == 2
                              matrix `boot_results'[`b',6] = r(mean)
                          }
                          
                          matrix `boot_results'[`b',1] = rho_c_bs
                          matrix `boot_results'[`b',2] = rho_hat_bs
                      }
                      
                      // Reload working data after bootstrap
                      use `working_data', clear
                      
                      **** STEP 5: Calculate means and standard errors ****
                      di ""
                      di "Bootstrap completed."
                      di ""
                      di "=== Bootstrap Poverty Transition Shares ==="
                      
                      preserve
                      clear
                      svmat `boot_results', names(col)
                      
                      foreach var in rho_c rho_partial P11 P10 P01 P00 {
                          sum `var', meanonly
                          scalar mean_`var' = r(mean)
                          scalar se_`var' = r(sd)
                      }
                      
                      di "Stayed Poor (P11):       " %5.1f mean_P11*100 "%  (SE: " %4.2f se_P11*100 "%)"
                      di "Escaped Poverty (P10):   " %5.1f mean_P10*100 "%  (SE: " %4.2f se_P10*100 "%)"
                      di "Fell into Poverty (P01): " %5.1f mean_P01*100 "%  (SE: " %4.2f se_P01*100 "%)"
                      di "Stayed Non-poor (P00):   " %5.1f mean_P00*100 "%  (SE: " %4.2f se_P00*100 "%)"
                      di ""
                      
                      // Save results if requested
                      if "`saveresults'" != "" {
                          export excel using "`saveresults'", firstrow(variables) replace
                          di "Results saved to `saveresults'"
                          di ""
                      }
                      
                      restore
                      
                      **** Return results ****
                      return scalar rho_c = mean_rho_c
                      return scalar rho_partial = mean_rho_partial
                      return scalar P11_mean = mean_P11
                      return scalar P11_se = se_P11
                      return scalar P10_mean = mean_P10
                      return scalar P10_se = se_P10
                      return scalar P01_mean = mean_P01
                      return scalar P01_se = se_P01
                      return scalar P00_mean = mean_P00
                      return scalar P00_se = se_P00
                      
                  end
                  
                  **** EXAMPLE USAGE ****
                  synpanel_point, ///
                      df_round1("round1.dta") ///
                      df_round2("round2.dta") ///
                      x_cols("gender age1 age2 age3 age4") ///
                      cohort_cols("gender age1 age2 age3 age4") ///
                      dep_var_round1("rcons") ///
                      dep_var_round2("rcons") ///
                      povline_r1("pline_7") ///
                      povline_r2("pline") ///
                      logtransform ///
                      n_bootstrap(5) ///
                      seed(42) ///
                      saveresults("synpanel_results.xlsx")

                  Comment


                  • #10
                    This is better, I think. Not a command but shows what what. It's a replicate of synthetic_panel in Python. No warranty.

                    Code:
                    *******************************************************
                    * Synthetic panel – Python-style estimate_transitions
                    *******************************************************
                    
                    clear all
                    set more off
                    
                    ***************************************
                    * USER INPUTS
                    ***************************************
                    local r1 "round1.dta"
                    local r2 "round2.dta"
                    
                    local dep1  rcons
                    local dep2  rcons
                    local pline1 pline_7
                    local pline2 pline
                    
                    local X gender age1 age2 age3 age4
                    local B 5      // number of bootstrap reps
                    
                    *******************************************************
                    * 1. ROUND 1: prepare data
                    *******************************************************
                    use "`r1'", clear
                    
                    * log transform (Python: log_transform=True)
                    gen double y1 = ln(`dep1')
                    gen double p1 = ln(`pline1')
                    
                    * create cohort from X (Python concatenates; group() is equivalent)
                    egen long cohort = group(`X')
                    
                    tempfile R1
                    save `R1', replace
                    
                    *******************************************************
                    * 2. ROUND 2: prepare data
                    *******************************************************
                    use "`r2'", clear
                    
                    gen double y2 = ln(`dep2')
                    gen double p2 = ln(`pline2')
                    
                    egen long cohort = group(`X')
                    
                    tempfile R2
                    save `R2', replace
                    
                    *******************************************************
                    * 3. PRECOMPUTE ITEMS THAT DON'T CHANGE IN BOOTSTRAP
                    *******************************************************
                    
                    * 3a. Covariance of X in round 2 (df_round2[x_cols].cov())
                    use `R2', clear
                    corr `X', covariance
                    matrix covX = r(C)
                    
                    * 3b. SD of y1 in round 1 (ystd_round1)
                    use `R1', clear
                    quietly summ y1
                    scalar sd_y1 = r(sd)
                    
                    * 3c. Regression in round 1: y1 ~ X (no constant)
                    reg y1 `X', noconstant
                    matrix b1 = e(b)'      //  k x 1 vector of betas
                    scalar sd_e1 = e(rmse) // residual SD in round 1
                    
                    * 3d. Regression in round 2: y2 ~ X (no constant)
                    use `R2', clear
                    reg y2 `X', noconstant
                    matrix b2 = e(b)'      //  k x 1 vector of betas
                    scalar sd_e2 = e(rmse) // residual SD in round 2
                    
                    * 3e. mm = coeffs_round1' * cov_matrix * coeffs_round2
                    matrix mm_mat = b1' * covX * b2
                    scalar mm = mm_mat[1,1]
                    
                    * 3f. cohort means of y1 (constant across bootstrap)
                    use `R1', clear
                    collapse (mean) y1_mean = y1, by(cohort)
                    tempfile R1_mean
                    save `R1_mean', replace
                    
                    * 3g. cohort means of log poverty line in round 1
                    use `R1', clear
                    collapse (mean) p1_mean = p1, by(cohort)
                    tempfile PL1
                    save `PL1', replace
                    
                    *******************************************************
                    * 4. BOOTSTRAP LOOP
                    *******************************************************
                    tempname OUT
                    matrix OUT = J(`B', 6, .)
                    matrix colnames OUT = rho_c rho_partial P11 P10 P01 P00
                    
                    set seed 42
                    
                    forvalues b = 1/`B' {
                    
                        di as txt "Bootstrap iteration `b' of `B'"
                    
                        **********************************************
                        * 4.1 Draw ONE bootstrap sample from round 2
                        **********************************************
                        use `R2', clear
                        bsample   // sample size = N(R2), with replacement
                    
                        **********************************************
                        * 4.2 Merge in cohort-level p1_mean from round 1
                        **********************************************
                        merge m:1 cohort using `PL1', nogen
                    
                        **********************************************
                        * 4.3 Compute fitted values yhat1 (round1 betas)
                        *     and yhat2 (round2 betas) on this sample
                        **********************************************
                        tempvar yhat1 yhat2
                        gen double `yhat1' = 0
                        gen double `yhat2' = 0
                    
                        local k : word count `X'
                        forvalues j = 1/`k' {
                            local v : word `j' of `X'
                            quietly replace `yhat1' = `yhat1' + b1[`j',1] * `v'
                            quietly replace `yhat2' = `yhat2' + b2[`j',1] * `v'
                        }
                    
                        **********************************************
                        * 4.4 Compute rho_c on cohort means
                        *      (mean y1 from R1, mean y2 from this sample)
                        **********************************************
                        preserve
                            collapse (mean) y2_mean = y2, by(cohort)
                            merge 1:1 cohort using `R1_mean', nogen
                            corr y1_mean y2_mean
                            scalar rho_c = r(rho)
                        restore
                    
                        **********************************************
                        * 4.5 Compute sd_y2 on THIS bootstrap sample
                        **********************************************
                        quietly summ y2
                        scalar sd_y2 = r(sd)
                    
                        **********************************************
                        * 4.6 Compute rho_partial (Python formula)
                        **********************************************
                        scalar rho_partial = (rho_c * sd_y1 * sd_y2 - mm) / (sd_e1 * sd_e2)
                    
                        * Clip to (-0.9999, 0.9999) as in Python
                        if (rho_partial > 0.9999) scalar rho_partial = 0.9999
                        if (rho_partial < -0.9999) scalar rho_partial = -0.9999
                    
                        **********************************************
                        * 4.7 Transition probabilities via binormal CDF
                        **********************************************
                        tempvar z1 z2
                        gen double `z1' = (p1_mean - `yhat1') / sd_e1
                        gen double `z2' = (p2       - `yhat2') / sd_e2
                    
                        tempvar P11 P10 P01 P00
                    
                        * Stayed Poor (P11)
                        gen double `P11' = binormal(`z1', `z2',  rho_partial)
                    
                        * Escaped Poverty (P10)
                        gen double `P10' = binormal(`z1', -`z2', -rho_partial)
                    
                        * Fell into Poverty (P01)
                        gen double `P01' = binormal(-`z1', `z2', -rho_partial)
                    
                        * Stayed Non-poor (P00)
                        gen double `P00' = binormal(-`z1', -`z2', rho_partial)
                    
                        **********************************************
                        * 4.8 Store bootstrap means in matrix OUT
                        **********************************************
                        quietly summ `P11'
                        matrix OUT[`b',3] = r(mean)
                    
                        quietly summ `P10'
                        matrix OUT[`b',4] = r(mean)
                    
                        quietly summ `P01'
                        matrix OUT[`b',5] = r(mean)
                    
                        quietly summ `P00'
                        matrix OUT[`b',6] = r(mean)
                    
                        matrix OUT[`b',1] = rho_c
                        matrix OUT[`b',2] = rho_partial
                    }
                    
                    *******************************************************
                    * 5. DISPLAY RAW BOOTSTRAP DRAWS
                    *******************************************************
                    mat li OUT
                    
                    *******************************************************
                    * 6. SUMMARY: MEANS AND SEs
                    *******************************************************
                    capture program drop dotable
                    program dotable
                    svmat OUT, names(col)
                    di `"{hline 50}"'
                    di _col(10) "rho"
                    di `"{hline 50}"'
                    qui summ rho_c
                    di  "rho_c mean  = " %9.4f r(mean)
                    qui summ rho_partial
                    di "rho_partial mean  = " %9.4f r(mean)
                    di `"{hline 50}"'
                    di "Bootstrap Poverty Transition Shares"
                    qui summ P11
                    di "Stayed Poor (P11): " %4.1f 100*r(mean) "% (SE: " %4.2f 100*r(sd) "%)"
                    qui summ P10
                    di "Stayed Poor (P10): " %4.1f 100*r(mean) "% (SE: " %4.2f 100*r(sd) "%)"
                    qui summ P01
                    di "Stayed Poor (P01): " %4.1f 100*r(mean) "% (SE: " %4.2f 100*r(sd) "%)"
                    qui summ P00
                    di "Stayed Poor (P00): " %4.1f 100*r(mean) "% (SE: " %4.2f 100*r(sd) "%)"
                    di `"{hline 50}"'
                    di "By Bootstrap"
                    di `"{hline 50}"'
                    di _col(10) "rho_c" _col(20) "rho_par" _col(30) "Stay Poor" _col(40) "Esc Pov"
                    di _col(32) "(P11)" _col(41) "(P10)"
                    di `"{hline 50}"'
                    local B = rowsof(OUT)
                    forv i = 1/`B' {
                        di `i' _col(10) %6.4f OUT[`i',1] _col(20) %6.4f OUT[`i',2] _col(30) %6.4f OUT[`i',3] _col(40) %6.4f OUT[`i',4]  
                    }
                    di `"{hline 50}"'
                    
                    end
                    
                    dotable

                    Comment


                    • #11
                      HTML Code:
                              
                      rho
                              
                      rho_c mean  =   -0.1468
                      rho_partial mean  =   -0.1221
                              
                      Bootstrap Poverty Transition Shares
                      Stayed Poor (P11):  2.3% (SE: 1.07%)
                      Stayed Poor (P10): 17.5% (SE: 1.16%)
                      Stayed Poor (P01): 13.4% (SE: 1.00%)
                      Stayed Poor (P00): 66.8% (SE: 1.03%)
                              
                      By Bootstrap
                              
                                rho_c    rho_par   Stay Poor    Esc Pov
                                                     (P11)       (P10)
                      
                      1        -0.3110   -0.2808   0.0143    0.1904
                      2        0.0486    0.0746    0.0350    0.1653
                      3        -0.3919   -0.3745   0.0098    0.1849
                      4        -0.0638   -0.0386   0.0274    0.1690
                      5        -0.0159   0.0090    0.0294    0.1666
                              

                      Comment


                      • #12
                        data files
                        Attached Files

                        Comment


                        • #13
                          thanks for your support Prof. George Ford , however i am calculating without bootstraps, using the code you provided but i get "P11=100%". This result is not correct, how to fix it?
                          clear all
                          set more off
                          local r1 "new.dta"
                          local r2 "new_2.dta"
                          local dep1 ln_ic
                          local dep2 ln_ic
                          local pline1 pline_7
                          local pline2 pline
                          local X sex yob edu dwelling provine
                          use "`r1'", clear
                          gen double y1 = `dep1'
                          gen double p1 = `pline1'
                          egen long cohort = group(`X')
                          tempfile R1
                          save `R1', replace
                          use "`r2'", clear
                          gen double y2 = `dep2'
                          gen double p2 = `pline2'
                          egen long cohort = group(`X')
                          tempfile R2
                          save `R2', replace
                          use `R2', clear
                          corr `X', covariance
                          matrix covX = r(C)
                          use `R1', clear
                          quietly summ y1
                          scalar sd_y1 = r(sd)
                          reg y1 `X', noconstant
                          matrix b1 = e(b)'
                          scalar sd_e1 = e(rmse)
                          use `R2', clear
                          reg y2 `X', noconstant
                          matrix b2 = e(b)'
                          scalar sd_e2 = e(rmse)
                          matrix mm_mat = b1' * covX * b2
                          scalar mm = mm_mat[1,1]
                          use `R1', clear
                          collapse (mean) y1_mean = y1, by(cohort)
                          tempfile R1_mean
                          save `R1_mean', replace
                          use `R1', clear
                          collapse (mean) p1_mean = p1, by(cohort)
                          tempfile PL1
                          save `PL1', replace
                          use `R2', clear
                          collapse (mean) y2_mean = y2, by(cohort)
                          merge 1:1 cohort using `R1_mean', nogen
                          corr y1_mean y2_mean
                          scalar rho_c_point = r(rho)
                          use `R2', clear
                          quietly summ y2
                          scalar sd_y2_point = r(sd)
                          scalar rho_partial_point = (rho_c_point * sd_y1 * sd_y2_point - mm) / (sd_e1 * sd_e2)
                          use `R2', clear
                          merge m:1 cohort using `PL1', nogen
                          tempvar yhat1 yhat2
                          gen double `yhat1' = 0
                          gen double `yhat2' = 0
                          local k : word count `X'
                          forvalues j = 1/`k' {
                          local v : word `j' of `X'
                          quietly replace `yhat1' = `yhat1' + b1[`j',1] * `v'
                          quietly replace `yhat2' = `yhat2' + b2[`j',1] * `v'
                          }
                          tempvar z1 z2
                          gen double `z1' = (p1_mean - `yhat1') / sd_e1
                          gen double `z2' = (p2 - `yhat2') / sd_e2
                          gen double P11_point = binormal(`z1', `z2', rho_partial_point)
                          gen double P10_point = binormal(`z1', -`z2', -rho_partial_point)
                          gen double P01_point = binormal(-`z1', `z2', -rho_partial_point)
                          gen double P00_point = binormal(-`z1', -`z2', rho_partial_point)
                          di `"{hline 50}"'
                          di as txt "POINT ESTIMATES"
                          di `"{hline 50}"'
                          di as txt "rho_c: " as result %9.4f rho_c_point
                          di as txt "rho_partial: " as result %9.4f rho_partial_point
                          di `"{hline 50}"'
                          di as txt "Poverty Transition Share"
                          quietly summ P11_point
                          di as txt "(P11): " as result %4.1f 100*r(mean) "%"
                          quietly summ P10_point
                          di as txt "(P10): " as result %4.1f 100*r(mean) "%"
                          quietly summ P01_point
                          di as txt "(P01): " as result %4.1f 100*r(mean) "%"
                          quietly summ P00_point
                          di as txt "(P00): " as result %4.1f 100*r(mean) "%"
                          di `"{hline 50}"

                          Comment


                          • #14
                            sorry for my mistake above code, I fixed it, but I have some questions, can I text you?
                            Last edited by LEE CHAN; 16 Nov 2025, 12:30.

                            Comment


                            • #15
                              That's a bug. I was getting that on occasion when converting the Python code. Are you using the exact same code? You should be able to make a few changes at the top for your variables and datasets and go.

                              Comment

                              Working...
                              X