Weight Distribution Approach to trim excessive Weights

[Andy Daniel]

Hi,

I'm currently applying some weighting procedures to panel data. One of the common issues in weighting is the occurence of excessive weights (>40) which should be trimmed to a lower level. Here is where trouble starts for me. One of the major (in in some opionions only) ways to deal with this problem is the Weight Distribution Approach by Potter (1990). The theoretical Design of this approach is almost clear to me but I'm not able to implement it in Stata. So my question is: is there an .ado which applies the weight distribution approach to weights?

Many thanks for your help!!

Best regards,

Andy

[Stephen Jenkins]

Welcome to Statalist! Please take a few moments to read the Forum FAQ (hit the black bar at top of the page), in particular all the entries recommending how to frame questions in a way that will maximize the chances of getting a helpful response. My quick reactions: (1) give full and complete references ("Potter (1990)" fails this test) -- why should readers be assumed to know what you're referring to? See the FAQ on this. (2) why can't you simply manually top-code outlier weights in some sort of systematic manner. (I mean that if the weight is greater than a critical cut-off, then reset the weight to the cut-off value.) The producers of many large household surveys do this routinely before releasing their data! You haven't explained the principles underlying Potter's work, and how it differs from the common practice I refer to, so you are unlikely to solicit many answers -- unless you provide the information.

[Andy Daniel]

Dear Stephen,

thank you for your advice and sorry for leaving out the references. You are complety right, it makes no sense to quote without a complete reference.

The problem I have with simple top coding is that you lose the variance of the weights which are situated above the cut-off-criteria (with a cut-off-value of 40, weights with the values 41 and 80 are treated as alike). Potters apporach assumes the weights to follow an inverse beta distribution. Thus the parameters of the distribution are estimated using the weights. To trim the excessive weights, a trimming level is defined and computed (e.g. occurence probability 0,5%) and all weights in excess of this level are trimmed to the trimming level (very similiar to top-coding) and the excess is distributed among the untrimmed weights (for me this is the key feature of the approach). Then the distribution is estimated again with the trimmed weights resulting in a new trimming level, then the weights are compared to the newly computed trimming level and if an excess occurs the trimming and redistribution is done again. This iterative process continues until no excessive weights occur.

I (mistakenly) thought this approach was a very common procedure to trim weights and thus I guessed there might already be an .ado at hand.

Reference:
Potter, F. J. (1990). A study of procedures to identify and trim extreme sampling weights. In American Statistical Association (Ed.), Proceedings of the Survey Research Methods Section pp. 225. --> http://www.amstat.org/sections/srms/...s/1990_034.pdf

Sorry again for the quick shot in my frist post.

Best Regards,

Andy

[Stephen Jenkins]

Thanks for the reference and link. I don't think Potter's procedure is very common. I am not a survey specialist, but here's my take.

I think Potter's bespoke nature is likely to count against it. Put differently, I suspect that producers of general purpose surveys are unlikely to implement such a procedure routinely -- they produce one or two sets of all-purpose weights. If one has a specialist survey undertaken to estimate some particular statistic(s), then the approach may be different. This is consistent with Potter's message: he stresses that choice of trimming procedure has to trade off issues of potential increase in bias versus reduction in sampling variance (and thence overall MSE). But to do the calculations in a particular context, you have to choose the statistic or statistics for which you calculate the MSE. In a general purpose survey, it's unclear ex ante what the statistic(s) is. Another off-the-cuff remark is: why "inverse beta" distribution and not some other? And why trim top 0.5% rather than say top 0.75% or top 0.1%?

If you really did want to implement Potter's approach, then I think it's do-able with some coding by you. First, find Stata code for "inverse beta distribution". I don't know whether this is actually the "inverted beta distribution" (as described in Wikipedia pages) or the "inverse beta distribution". In the former case, it appears that the distribution is a special case of the generalised beta distribution of the second kind -- so look at gb2fit and gb2lfit on SSC (and note that you can use constraint to constrain parameters). For the latter case, there is betafit on SSC. After fitting, you can do the weight spreading according to your chosen algorithm, iterate the procedure, etc., through to convergence in the way you describe.

Another perspective is that "outlier" weights could be examined like outliers in other variable. I would closely inspect them to discover what factors are associated with being an apparent "outlier" -- to see whether outlier-ness is just wacky or somehow genuine (this is a judgement call of course; one can never know the "true" answer). This recommendation is also consistent with Potter's conclusions which discuss whether or not to implement trimming etc: note his remarks about correlations between weights and other variables.

[Andy Daniel]

Dear Stephen,

thank you very much for your comprehensive comment. I get your point concering the MSE but I think your argumentation is also valid for top-coding, since top-coding follows the same goals as any other trimming procedure. Consequently trimming would be obsolet in general purpose surveys because you don't know which statistics the data-users will use and if your chosen trimming procedure was benefitial to this statistics. I don't get your point why this is especially true for Potter's Weight-Distribution-Approach since he don't uses the MSE in the weighting procedure. The Taylor Series Procedures, which is also proposed in Potter's paper, is an example of an approach where the MSE is used for weighting. Here I agree with you that this method is more appropriate in such cases where the the data set is gathered for a special purpose and the analytical framework is already set. I also agree with your objection that the decsion for the trimming level is (like top-coding) a judgement call. I think the advantage of Potter's apporach resides in the redistribution of the excess above the chosen level. Why Potter has chosen a derivate of the beta distribution in favor of other probability functions isn't clear to me either.

Thanks for your suggestions concering the implementation of the approach. I think I will try to implement it and compare it to top-coding. But you are right, prior to the implementation I should check the necessity of trimming in the frist place.

Best regards,

Andy

[Steve Samuels]

For a more recent approach with other references, see: Elliot, 2008. SUDAAN has a built-in procedure: http://www.rti.org/sudaan/onlinehelp...0Procedure.htm. Sampling weights can't be trimmed in isolation, as they will often be further post-stratified, raked, or calibrated. Calibrated weights can be unacceptably small. See Brewer (1999, 2002, p. 133); Kott (2011); Sarndal ( 2011).


References:

Elliott, M. R. (2008). Model Averaging Methods for Weight Trimming. Journal of Official Statistics, 24(4), 517–540.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2783643/


Brewer, Ken. 2002. Combined survey sampling inference : weighing Basu’s elephants. London ; New York: Arnold Publishers; Distributed in the United States of America by Oxford University Press.

Kott, P. S. (2011). A NEARLY PSEUDO-OPTIMAL METHOD FOR KEEPING CALIBRATION WEIGHTS FROM FALLING BELOW UNITY IN THE ABSENCE OF NONRESPONSE OR FRAME ERRORS. Pakistan Journal of Statistics, 27(4)
http://pakjs.com/journals/27(4)/27(4)5.pdf

Särndal, Carl-Erik. "Pak. J. Statist. 2011 Vol. 27 (4), 359-370 COMBINED INFERENCE IN SURVEY SAMPLING." Pak. J. Statist 27.4 (2011): 359-370. http://pakjs.com/journals/27(4)/27(4)3.pdf

[Andy Daniel]

Hi Steve,

sorry for my very late response on your post and thank you very much for the additional literature. I think that will help me a lot. When I got a final solution for my case, I will give a short report in this thread.

best regards

Andy

[Steve Samuels]

You're welcome, Andy. I should have pointed out that small weights are less harmful than large extreme weights.