Data collection: estimation of a respondent burden in a statistical survey

[Sergiy Radyakin]

Dear All,

sorry for an off-topic question not related directly to Stata, but I would like to ask the experts here for an advice on respondent paperwork burden estimation methodology.

Respondent burden estimation is sometimes reported on standardized forms. See for example here:
https://en.wikipedia.org/wiki/Paperwork_Reduction_Act

A particular form or questionnaire may be estimated to carry, say, 2 hours "respondent burden":
illustration

My question is the following: How is the paperwork burden estimated? Or more specifically, is it estimated "directly" or "indirectly"?

• by directly I mean something like the following: N persons are given the form, time is measured for each one to complete it, then averaged;
• by indirectly I mean something like the following: this form contains 30 open ended questions (each is assumed 2 minutes) and 60 yes/no questions (each is assumed 30 seconds) hence the total burden is 30x2+60x0.5=90 minutes or 1.5 hour.

In both cases I see significant savings from learning-by-doing: the first time I am filling out the form of any kind will require me way more time than the 100th. In estimating the respondent burden, is there any assumption about the position of the respondent on the learning curve? (e.g for first-timer? for profi?)

If the "indirect" method is applied: Is there any publicly available resource with "costs" per question? per skip condition, etc.

Thank you very much, Sergiy Radyakin

[Joseph Coveney]

There might not be a government-wide uniform, formal, explicitly defined method; 5 CFR 1320.8(a)4 calls for "A specific, objectively supported estimate of burden . . .", but I didn't find anywhere in that title, chapter, subchapter and part any prescriptive passage regarding how that is to be done.

One agency has this on how to go about it:

For rule-related ICRs [Information Collection Requests], burden estimates for each activity should be extracted from the corresponding EA [Economic Analysis]. If EA estimates are not available, base your estimates on informal consultations with nine or fewer respondents or on your experience with similar collections. Remember to list all consultations in Section 3(c). You should not conduct special surveys to estimate burden. If you plan to pretest a questionnaire or other reporting form, ask respondents at the time of the pretest debriefing for burden information. Your burden estimates (either newly developed or estimates from a previous ICR, if your ICR is a renewal) should be summarized in the required Federal Register notice. Public comments from actual respondents should help you develop more accurate burden estimates. (Emphasis in original.)

[Sergiy Radyakin]

Dear Joseph,

thank you very much for taking time to look into the issue. My fear is that there is no clear manual and a worked out example, but rather recommendations and advice on what must be accounted for. I have since looked into:
http://ec.europa.eu/eurostat/documen...NSE-BURDEN.pdf
http://www.amstat.org/meetings/ices/...ers/302289.pdf
and some of the articles referenced from there, but couldn't find the answer to my question.

I am also puzzled by the "fewer than 9" clause in the text you referred to, as 1 is fewer than 9, can I then always equate my estimate to my own opinion? Even if we take 9, some of the surveys contain a significant amount of branching and optional sections, and 9 is nowhere near to the number of possible combinations.

If anyone can come up with a good source for estimation of time and complexity of surveys, I would be glad to receive a hint. If there is another forum or group that is closer to the topic, I would appreciate if I can be redirected there.

Thank you, Sergiy Radyakin