[Nick Cox]

http://www.statalist.org/forums/help

(This is the document that you were asked to read before starting a thread.)

[William Lisowski]

I'm not fond of "piling on", but I referred you to the Statalist FAQ in post #12 of this earlier topic that you started.

The more you help others understand your problem, the more likely others are to be able to help you solve your problem. And conversely, the more trouble others have understanding your problem, despite repeated referrals to guidance on how to help others help you, the less likely they are to step forward and assist with subsequent problems.

[Inigo Sanchez]

[CODE]----------------------- copy starting from the next line -----------------------

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(var1 var2 second1 second2)
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
   .0025    .0025  .047837  .002571
   .0025    .0025  .027636  .029622
   .0025    .0025  .068782 -.009517
   .0025    .0025  .033362  .014938
   .0025    .0025  .004348  .017665
   .0025    .0025  .055877  .027512
   .0025    .0025  .015686 -.001496
    .005     .005  .004362  .017257
    .005     .005  .015731 -.001947
     .01      .01  .014896 -.002407
   .0025    .0025  .005435  .019725
   .0025    .0025  .011764  .019286
       .        .        .        .
       .        .        .        .
       .        .  .038386  .012175
       .        .   .03831  .011674
       .        .  .041368  .034724
       .        .  .041368  .034724
       .        .  .038281  .010944
       .        .  .041688  .035085
     .01      .01  .056465  .041938
       .      .01  .056465  .041938
   .0025    .0025  .057815  .041923
     .01      .01  .058387  .020533
     .01      .01  .057492  .021397
   .0025    .0025  .058331  .021863
     .01      .01  .027539  .010709
     .01      .01  .027539  .010709
   .0025    .0025  .027848  .011295
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .

The data is for the first, second, 121th and 122th variables

[Nick Cox]

Thanks for finding out how to use dataex (although the example is truncated; the code needs a closing end to run successfully).

However, the data example shows

1. No string variables.

2. Correspondingly no instances of values that are problematic as earlier reported.

[William Lisowski]

In post #1 you wrote

some observations are null (i.e. they display "-")

In post #9 you wrote

nothing has changed, there are still "-" null values

But what we see in your sample data are standard Stata missing values represented by

. rather than -

It appears that you have misunderstood what you saw, most likely from the very start, since the data extract you provided shows that your data are numeric and you did not indicate that you ran the destring command after running Nick's code in post #8.

[Inigo Sanchez]

Yes sorry, I misunderstood it totally. Thank you again.

[Inigo Sanchez]

Referring back to #6, when introducing the following code I get an error. To be more specific, Stata tells "invalid syntax":

 forvalues i / 1/120 {
egen mean_decile_second`i' = mean(second`i'), by(decile_var`i')
}

Which could be the reason? Thank you in advance.

[Nick Cox]

That should begin

forvalues i = 1/120 {

as a study of the help for forvalues will confirm.

Also, you've put all the syntax on one line. That won't work. The original linebreaks should be respected.

[Inigo Sanchez]

Also, when entering the following code, Stata tells me that it is not able to find mean_decile_second1 and so on so forth. The code comes from #6:

  
 reshape long decile_ mean_decile_, i(obs_no) j(varname) string

[William Lisowski]

That suggests to me that perhaps you have yet to make the change suggested in #23, since that loop is the code that generates the variables that it cannot find. The error message you cite isn't exactly what I would have expected. From the FAQ

Say exactly what you typed and exactly what Stata typed (or did) in response. N.B. exactly!

You should copy from the Stata Results window both the command and the message from Stata.

[Inigo Sanchez]

I have entered the code correctly as suggested in #23. Here is the error:

reshape long decile_ mean_decile_, i(obs_no) j(varname) string
(note: j = second1 second10 second100 second101 second102 second103 second104 second105 second106 second107 second108 second109 second11 s
> econd110 second111 second112 second113 second114 second115 second116 second117 second118 second119 second12 second120 second13 second14 
> second15 second16 second17 second18 second19 second2 second20 second21 second22 second23 second24 second25 second26 second27 second28 se
> cond29 second3 second30 second31 second32 second33 second34 second35 second36 second37 second38 second39 second4 second40 second41 secon
> d42 second43 second44 second45 second46 second47 second48 second49 second5 second50 second51 second52 second53 second54 second55 second5
> 6 second57 second58 second59 second6 second60 second61 second62 second63 second64 second65 second66 second67 second68 second69 second7 s
> econd70 second71 second72 second73 second74 second75 second76 second77 second78 second79 second8 second80 second81 second82 second83 sec
> ond84 second85 second86 second87 second88 second89 second9 second90 second91 second92 second93 second94 second95 second96 second97 secon
> d98 second99 var1 var10 var100 var101 var102 var103 var104 var105 var106 var107 var108 var109 var11 var110 var111 var112 var113 var114 v
> ar115 var116 var117 var118 var119 var12 var120 var13 var14 var15 var16 var17 var18 var19 var2 var20 var21 var22 var23 var24 var25 var26 
> var27 var28 var29 var3 var30 var31 var32 var33 var34 var35 var36 var37 var38 var39 var4 var40 var41 var42 var43 var44 var45 var46 var47 
> var48 var49 var5 var50 var51 var52 var53 var54 var55 var56 var57 var58 var59 var6 var60 var61 var62 var63 var64 var65 var66 var67 var68 
> var69 var7 var70 var71 var72 var73 var74 var75 var76 var77 var78 var79 var8 var80 var81 var82 var83 var84 var85 var86 var87 var88 var89 
> var9 var90 var91 var92 var93 var94 var95 var96 var97 var98 var99)
(note: decile_second1 not found)
(note: decile_second10 not found)
(note: decile_second100 not found)
(note: decile_second101 not found)
(note: decile_second102 not found)
(note: decile_second103 not found)
(note: decile_second104 not found)
(note: decile_second105 not found)
(note: decile_second106 not found)
(note: decile_second107 not found)
(note: decile_second108 not found)
(note: decile_second109 not found)
(note: decile_second11 not found)
(note: decile_second110 not found)
(note: decile_second111 not found)
(note: decile_second112 not found)
(note: decile_second113 not found)
(note: decile_second114 not found)
(note: decile_second115 not found)
(note: decile_second116 not found)

[Clyde Schechter]

OK. I have revised the approach slightly. The distribution of missing values in your data makes this rather complicated, as does the structure overall. But the following works with the sample data you have posted:

set more off
* Example generated by -dataex-. To install: ssc install dataex
clear
input float(var1 var2 second1 second2)
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
   .0025    .0025  .047837  .002571
   .0025    .0025  .027636  .029622
   .0025    .0025  .068782 -.009517
   .0025    .0025  .033362  .014938
   .0025    .0025  .004348  .017665
   .0025    .0025  .055877  .027512
   .0025    .0025  .015686 -.001496
    .005     .005  .004362  .017257
    .005     .005  .015731 -.001947
     .01      .01  .014896 -.002407
   .0025    .0025  .005435  .019725
   .0025    .0025  .011764  .019286
       .        .        .        .
       .        .        .        .
       .        .  .038386  .012175
       .        .   .03831  .011674
       .        .  .041368  .034724
       .        .  .041368  .034724
       .        .  .038281  .010944
       .        .  .041688  .035085
     .01      .01  .056465  .041938
       .      .01  .056465  .041938
   .0025    .0025  .057815  .041923
     .01      .01  .058387  .020533
     .01      .01  .057492  .021397
   .0025    .0025  .058331  .021863
     .01      .01  .027539  .010709
     .01      .01  .027539  .010709
   .0025    .0025  .027848  .011295
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
       .        .        .        .
end


foreach v of varlist var1-var2 {
    xtile decile_`v' = `v', nq(10)
    replace decile_`v' = 11-decile_`v'
}

//    THERE ARE OBSERVATIONS WHERE VAR1 OR VAR2 IS MISSING BUT SECOND1 (RESP. SECOND2)
//    ARE NOT.  THESE OBSERVATIONS DO NOT CONSTITUTE A LEGITIMATE "DECILE", BUT
//    AS THEY OCCUR IN DIFFERENT OBSERVATIONS, WE CANNOT JUST DROP THEM.  THE MISSING
//    VALUES IN THE DECILE VARIABLES CAUSE PROBLEMS WITH -RESHAPE-, SO TO WORK AROUND
//    THIS, RECODE THEM AS 99
mvencode decile_*, mv(99)

forvalues i = 1/2 {
    egen mean_decile_second`i' = mean(second`i') if decile_var`i' != 99, by(decile_var`i')
}

keep mean_decile_* decile_*
gen long obs_no = _n
reshape long decile_var mean_decile_second, i(obs_no) j(varnum)
mvdecode decile_, mv(99) // UNDO THE 99 ENCODING OF MISSING VALUES
drop obs_no
duplicates drop
drop if missing(decile_var)
reshape wide mean_decile_second, i(varnum) j(decile_)

You should be able to just change 2 to 120 in the appropriate places and use this.

Note: The above code ends up excluding all information about the second* variables when the corresponding var* variable has missing values. But that information might be useful. So if you want to retain that, remove the line of code that says -mvdecode decile_, mv(99) // UNDO THE 99 ENCODING OF MISSING VALUES-. The results will then contain an additional variable reflecting the "99'th" decile, which is an artificial way of saying that these are the mean values of the second* variables when the corresponding var* variable has a missing value.