AAPD average absolute point disagreement - a stupid and not optimized way to do that (but works)

[Riccardo Rasoni]

Hello everybody

I'm usually asking always questions, so I want to try to give my little contribute with something that, at least up to now, I didn't find here.

Dataset: Rotating panel dataset (but in this case works also with cross-sectional or normal panel); in my case I had w waves where the surveys were taken, every wave corresponds to a different month (sequential). In every survey I had i respondents (pretty unbalanced, since that they varied between a range of 9000-22000 per survey) that gave their personal value about a variable y (in my case, the expectation of inflation one year ahead)

My goal was to take the following measure called "AAPD average absolute point disagreement" and expressed in the formula:

\[
AAPD_{i,t+h|h}=\frac{1}{N_{t}-1}\sum_{j\neq i}^{}\left| y_{j,t+h|t}-y_{i,t+h|t} \right|
\]
whereby y_i represents a value of a variable of an individual, in a particular wave (in my case it was the expected inflation of the period ahead), y_j is the same but for every other individual but i.
N_t represents the number of id for every wave; in my case I expressed the t with the notation w.

The problem were essentially two:
1) how to create the value within the sum.
2) how to avoid to crash the computer.

I sorted it out with the following code:

use "your_data.dta", clear

levelsof wave, local(waves)
foreach w in `waves'{

use "your_data.dta", clear

levelsof wave, local(waves)
keep if wave==`w'

keep id wave y
bysort wave (id): gen id_num = _n

egen number= count(id_num), by(wave)


summarize number
scalar num=r(mean)
sort id_num

local num_a=num
forvalues i=1/`num_a'{
    display `i'
    gen temp_`i'= abs(y - y[_n+`i'])
    
    }
local num_b=num-1
forvalues i=1/`num_b'{
    replace temp_`i'=abs(y - y[_n+`i'-`num_a']) if missing(temp_`i')
    
}    
drop temp_`num_a'


egen AAPD_mean= rowmean(temp_1-temp_`num_b')


local num_b=num-1
forvalues i=1/`num_b'{
    drop temp_`i'
    }
    
save "data_wave_`w'.dta", replace

    }

It can be not very clear (I will answer to any possible question), though it works. The idea is to create different dataset that will be appended later on.
Every dataset refers only a particular wave.

I don't know how much it can be optimized, it shouldn't be so efficient, but at least is an easy way to provide the variable that proxys pretty good for individual disagreement in survey data.

Thanks
BR

Riccardo

[Nils Enevoldsen]

I believe that's exactly equivalent to the following code. It's not obvious to me if there's an elegant way to take out the nested loop.

use "your_data.dta", clear
levelsof wave, local(waves)

foreach w in `waves' {
    preserve
    keep if wave == `w'

    gen double AAPD_mean = 0
    forvalues i = 1/`=_N' {
        replace AAPD_mean = AAPD_mean + abs(y - y[`i'])/(`=_N' - 1)
    }
    
    sort id
    save "data_wave_`w'.dta", replace
    restore
}

Note that the variable named AAPD_mean might be better named simply AAPD — that is already a mean. I'm not sure if this already has another name. I would have expected something like "Mean Pairwise Distance".

[Riccardo Rasoni]

Thank you for your post. I will apply your suggestion and I'll let you know whether it works or not.
Anyway, I left "mean" just because, if anyone wants, you can also calculate the median, percentile etc... but maybe you're right, the proper index refers, originally, to a mean.

I believe that's exactly equivalent to the following code. It's not obvious to me if there's an elegant way to take out the nested loop.

use "your_data.dta", clear
levelsof wave, local(waves)

foreach w in `waves' {
preserve
keep if wave == `w'

gen double AAPD_mean = 0
forvalues i = 1/`=_N' {
replace AAPD_mean = AAPD_mean + abs(y - y[`i'])/(`=_N' - 1)
}

sort id
save "data_wave_`w'.dta", replace
restore
}

Note that the variable named AAPD_mean might be better named simply AAPD — that is already a mean. I'm not sure if this already has another name. I would have expected something like "Mean Pairwise Distance".